public inbox for [email protected]
help / color / mirror / Atom feedCorruption during WAL replay
94+ messages / 17 participants
[nested] [flat]
* Corruption during WAL replay
@ 2020-03-23 20:56 Teja Mupparti <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Teja Mupparti @ 2020-03-23 20:56 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Daniel Wood <[email protected]>
This is my *first* attempt to submit a Postgres patch, please let me know if I missed any process or format of the patch (I used this link https://wiki.postgresql.org/wiki/Working_with_Git<https://nam06.safelinks.protection.outlook.com/...; As reference)
The original bug reporting-email and the relevant discussion is here
https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de<https:/...;
https://www.postgresql.org/message-id/822113470.250068.1573246011818%40connect.xfinity.com<https:...;
https://www.postgresql.org/message-id/20191206230640.2dvdjpcgn46q3ks2%40alap3.anarazel.de<https:/...;
https://www.postgresql.org/message-id/[email protected]<https://nam06.safelinks.prote...;
The crux of the fix is, in the current code, engine drops the buffer and then truncates the file, but a crash before the truncate and after the buffer-drop is causing the corruption. Patch reverses the order i.e. truncate the file and drop the buffer later.
Warm regards,
Teja
Attachments:
[application/octet-stream] bug-fix-patch (10.4K, ../../BYAPR06MB6373BF50B469CA393C614257ABF00@BYAPR06MB6373.namprd06.prod.outlook.com/3-bug-fix-patch)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-03-24 09:18 Kyotaro Horiguchi <[email protected]>
parent: Teja Mupparti <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-24 09:18 UTC (permalink / raw)
To: [email protected]; +Cc: pgsql-hackers; [email protected]
Thanks for working on this.
At Mon, 23 Mar 2020 20:56:59 +0000, Teja Mupparti <[email protected]> wrote in
> This is my *first* attempt to submit a Postgres patch, please let me know if I missed any process or format of the patch
Welcome! The format looks fine to me. It would be better if it had a
commit message that explains what the patch does. (in the format that
git format-patch emits.)
> The original bug reporting-email and the relevant discussion is here
...
> The crux of the fix is, in the current code, engine drops the buffer and then truncates the file, but a crash before the truncate and after the buffer-drop is causing the corruption. Patch reverses the order i.e. truncate the file and drop the buffer later.
BufferAlloc doesn't wait for the BM_IO_IN_PROGRESS for a valid buffer.
I'm not sure it's acceptable to remember all to-be-deleted buffers
while truncation.
+ /*START_CRIT_SECTION();*/
Is this a point of argument? It is not needed if we choose the
strategy (c) in [1], since the protocol is aiming to allow server to
continue running after truncation failure.
[1]: https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
However, note that md truncates a "file" a non-atomic way. mdtruncate
truncates multiple files from the last segment toward the
beginning. If mdtruncate successfully truncated the first several
segments then failed, retaining all buffers triggers assertion failure
in mdwrite while buffer flush.
Some typos found:
+ * a backround task might flush them to the disk right after we
s/backround/background/
+ * saved list of buffers that were marked as BM_IO_IN_PRGRESS just
s/BM_IO_IN_PRGRESS/BM_IO_IN_PROGRESS/
+ * as BM_IO_IN_PROGRES. Though the buffers are marked for IO, they
s/BM_IO_IN_PROGRES/BM_IO_IN_PROGRESS/
+ * being dicarded).
s/dicarded/discarded/
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-03-30 23:31 Andres Freund <[email protected]>
parent: Kyotaro Horiguchi <[email protected]>
0 siblings, 3 replies; 94+ messages in thread
From: Andres Freund @ 2020-03-30 23:31 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; pgsql-hackers; [email protected]
Hi,
On 2020-03-24 18:18:12 +0900, Kyotaro Horiguchi wrote:
> At Mon, 23 Mar 2020 20:56:59 +0000, Teja Mupparti <[email protected]> wrote in
> > The original bug reporting-email and the relevant discussion is here
> ...
> > The crux of the fix is, in the current code, engine drops the buffer and then truncates the file, but a crash before the truncate and after the buffer-drop is causing the corruption. Patch reverses the order i.e. truncate the file and drop the buffer later.
>
> BufferAlloc doesn't wait for the BM_IO_IN_PROGRESS for a valid buffer.
I don't think that's true. For any of this to be relevant the buffer has
to be dirty. In which case BufferAlloc() has to call
FlushBuffer(). Which in turn does a WaitIO() if BM_IO_IN_PROGRESS is
set.
What path are you thinking of? Or alternatively, what am I missing?
> I'm not sure it's acceptable to remember all to-be-deleted buffers
> while truncation.
I don't see a real problem with it. Nor really a good alternative. Note
that for autovacuum truncations we'll only truncate a limited number of
buffers at once, and for most relation truncations we don't enter this
path (since we create a new relfilenode instead).
>
> + /*START_CRIT_SECTION();*/
> Is this a point of argument? It is not needed if we choose the
> strategy (c) in [1], since the protocol is aiming to allow server to
> continue running after truncation failure.
>
> [1]: https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
I think it's entirely broken to continue running after a truncation
failure. We obviously have to first WAL log the truncation (since
otherwise we can crash just after doing the truncation). But we cannot
just continue running after WAL logging, but not performing the
associated action: The most obvious reason is that otherwise a replica
will execute the trunction, but the primary will not.
The whole justification for that behaviour "It would turn a usually
harmless failure to truncate, that might spell trouble at WAL replay,
into a certain PANIC." was always dubious (since on-disk and in-memory
state now can diverge), but it's clearly wrong once replication had
entered the picture. There's just no alternative to a critical section
here.
If we are really concerned with truncation failing - I don't know why we
would be, we accept that we have to be able to modify files etc to stay
up - we can add a pre-check ensuring that permissions are set up
appropriately to allow us to truncate.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-03-31 07:36 Kyotaro Horiguchi <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 0 replies; 94+ messages in thread
From: Kyotaro Horiguchi @ 2020-03-31 07:36 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; pgsql-hackers; [email protected]
At Mon, 30 Mar 2020 16:31:59 -0700, Andres Freund <[email protected]> wrote in
> Hi,
>
> On 2020-03-24 18:18:12 +0900, Kyotaro Horiguchi wrote:
> > At Mon, 23 Mar 2020 20:56:59 +0000, Teja Mupparti <[email protected]> wrote in
> > > The original bug reporting-email and the relevant discussion is here
> > ...
> > > The crux of the fix is, in the current code, engine drops the buffer and then truncates the file, but a crash before the truncate and after the buffer-drop is causing the corruption. Patch reverses the order i.e. truncate the file and drop the buffer later.
> >
> > BufferAlloc doesn't wait for the BM_IO_IN_PROGRESS for a valid buffer.
>
> I don't think that's true. For any of this to be relevant the buffer has
> to be dirty. In which case BufferAlloc() has to call
> FlushBuffer(). Which in turn does a WaitIO() if BM_IO_IN_PROGRESS is
> set.
>
> What path are you thinking of? Or alternatively, what am I missing?
# I would be wrong with far low odds..
"doesn't" is overstated. Is there a case where the buffer is already
flushed by checkpoint? (If that is the case, dropping clean buffers at
marking truncate would work?)
> > I'm not sure it's acceptable to remember all to-be-deleted buffers
> > while truncation.
>
> I don't see a real problem with it. Nor really a good alternative. Note
> that for autovacuum truncations we'll only truncate a limited number of
> buffers at once, and for most relation truncations we don't enter this
> path (since we create a new relfilenode instead).
Thank you for the opinion. I agree to that.
> > + /*START_CRIT_SECTION();*/
>
> > Is this a point of argument? It is not needed if we choose the
> > strategy (c) in [1], since the protocol is aiming to allow server to
> > continue running after truncation failure.
> >
> > [1]: https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
>
> I think it's entirely broken to continue running after a truncation
> failure. We obviously have to first WAL log the truncation (since
> otherwise we can crash just after doing the truncation). But we cannot
> just continue running after WAL logging, but not performing the
> associated action: The most obvious reason is that otherwise a replica
> will execute the trunction, but the primary will not.
Hmm. If we allow PANIC on truncation failure why do we need to go on
the complicated steps? Wouldn't it enough to enclose the sequence
(WAL insert - drop buffers - truncate) in a critical section? I
believed that this project aims to fix the db-breakage on truncation
failure by allowing rollback on truncation failure?
> The whole justification for that behaviour "It would turn a usually
> harmless failure to truncate, that might spell trouble at WAL replay,
> into a certain PANIC." was always dubious (since on-disk and in-memory
> state now can diverge), but it's clearly wrong once replication had
> entered the picture. There's just no alternative to a critical section
> here.
Yeah, I like that direction.
> If we are really concerned with truncation failing - I don't know why we
> would be, we accept that we have to be able to modify files etc to stay
> up - we can add a pre-check ensuring that permissions are set up
> appropriately to allow us to truncate.
I think the question above is the core part of the problem.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-10 23:59 Teja Mupparti <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 1 reply; 94+ messages in thread
From: Teja Mupparti @ 2020-04-10 23:59 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Kyotaro Horiguchi <[email protected]>; +Cc: pgsql-hackers; [email protected] <[email protected]>
Thanks Andres and Kyotaro for the quick review. I have fixed the typos and also included the critical section (emulated it with try-catch block since palloc()s are causing issues in the truncate code). This time I used git format-patch.
Regards
Teja
________________________________
From: Andres Freund <[email protected]>
Sent: Monday, March 30, 2020 4:31 PM
To: Kyotaro Horiguchi <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: Corruption during WAL replay
Hi,
On 2020-03-24 18:18:12 +0900, Kyotaro Horiguchi wrote:
> At Mon, 23 Mar 2020 20:56:59 +0000, Teja Mupparti <[email protected]> wrote in
> > The original bug reporting-email and the relevant discussion is here
> ...
> > The crux of the fix is, in the current code, engine drops the buffer and then truncates the file, but a crash before the truncate and after the buffer-drop is causing the corruption. Patch reverses the order i.e. truncate the file and drop the buffer later.
>
> BufferAlloc doesn't wait for the BM_IO_IN_PROGRESS for a valid buffer.
I don't think that's true. For any of this to be relevant the buffer has
to be dirty. In which case BufferAlloc() has to call
FlushBuffer(). Which in turn does a WaitIO() if BM_IO_IN_PROGRESS is
set.
What path are you thinking of? Or alternatively, what am I missing?
> I'm not sure it's acceptable to remember all to-be-deleted buffers
> while truncation.
I don't see a real problem with it. Nor really a good alternative. Note
that for autovacuum truncations we'll only truncate a limited number of
buffers at once, and for most relation truncations we don't enter this
path (since we create a new relfilenode instead).
>
> + /*START_CRIT_SECTION();*/
> Is this a point of argument? It is not needed if we choose the
> strategy (c) in [1], since the protocol is aiming to allow server to
> continue running after truncation failure.
>
> [1]: https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
I think it's entirely broken to continue running after a truncation
failure. We obviously have to first WAL log the truncation (since
otherwise we can crash just after doing the truncation). But we cannot
just continue running after WAL logging, but not performing the
associated action: The most obvious reason is that otherwise a replica
will execute the trunction, but the primary will not.
The whole justification for that behaviour "It would turn a usually
harmless failure to truncate, that might spell trouble at WAL replay,
into a certain PANIC." was always dubious (since on-disk and in-memory
state now can diverge), but it's clearly wrong once replication had
entered the picture. There's just no alternative to a critical section
here.
If we are really concerned with truncation failing - I don't know why we
would be, we accept that we have to be able to modify files etc to stay
up - we can add a pre-check ensuring that permissions are set up
appropriately to allow us to truncate.
Greetings,
Andres Freund
Attachments:
[application/octet-stream] 0001-Wal-replay-corruption.patch (14.9K, ../../BYAPR06MB6373E32F65D12B7F958321AEABDE0@BYAPR06MB6373.namprd06.prod.outlook.com/3-0001-Wal-replay-corruption.patch)
download | inline diff:
From c744aa6046d4eef3660cab3c8d362686987b4ea6 Mon Sep 17 00:00:00 2001
From: Teja Mupparti <[email protected]>
Date: Sat, 21 Mar 2020 16:18:59 -0700
Subject: [PATCH] Wal replay corruption
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
A bug in Postgres which can cause redo recovery to fail(forever). There is a
very tiny timing window where an autovacuum completion, a checkpoint completion,
and an abnormal shutdown occur such that the checkpoint fails to write a dirty
page. Log replay requires the most up to date version of the page to be able to
replay the log record.
The abnormal shutdown can be an unrelated crash like that due to Out Of Memory
or a forced restart without a full shutdown checkpoint.
Repro details
1) Page on disk has empty LP 1, Insert into page LP 1
2) checkpoint START (Recovery REDO eventually starts here)
3) Delete all rows on the page (page is empty now)
4) Autovacuum kicks in and truncates the pages
DropRelFileNodeBuffers - Dirty page NOT written, LP 1 on disk
still empty
5) Checkpoint completes
6) Crash
7) smgrtruncate - Not reached (this is where we do the physical truncate)
Now the crash-recovery starts
Delete-log-replay (above step-3) reads page with empty LP 1 and the
delete fails with PANIC
Fix
===
1) Mark all the buffers as about-to-be-dropped
2) CacheInvalidateSmgr()
3) Truncate on filesystem level
if that fails, remove the about-to-be-dropped flags, in a PG_CATCH block
if that succeeds, fully remove the buffers to be dropped
Hackers discussion
===================
https://www.postgresql.org/message-id/822113470.250068.1573246011818%40connect.xfinity.com
https://www.postgresql.org/message-id/20191206230640.2dvdjpcgn46q3ks2%40alap3.anarazel.de
https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
https://www.postgresql.org/message-id/[email protected]
---
contrib/pg_visibility/pg_visibility.c | 1 +
src/backend/catalog/storage.c | 105 +++++++++++-----
src/backend/storage/buffer/bufmgr.c | 165 ++++++++++++++++++++++++++
src/backend/storage/smgr/smgr.c | 13 +-
src/include/storage/bufmgr.h | 4 +
5 files changed, 252 insertions(+), 36 deletions(-)
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index 0cd1160ceb..229ca8ca3f 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -398,6 +398,7 @@ pg_truncate_visibility_map(PG_FUNCTION_ARGS)
if (BlockNumberIsValid(block))
{
fork = VISIBILITYMAP_FORKNUM;
+ MarkTruncateBuffers(rel->rd_smgr->smgr_rnode, &fork, 1, &block);
smgrtruncate(rel->rd_smgr, &fork, 1, &block);
}
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index fddfbf1d8c..b071cb5666 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -276,45 +276,87 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
}
/*
- * We WAL-log the truncation before actually truncating, which means
- * trouble if the truncation fails. If we then crash, the WAL replay
- * likely isn't going to succeed in the truncation either, and cause a
- * PANIC. It's tempting to put a critical section here, but that cure
- * would be worse than the disease. It would turn a usually harmless
- * failure to truncate, that might spell trouble at WAL replay, into a
- * certain PANIC.
+ * 1) WAL-log the truncation of the file.
+ * 2) Discard buffers of the pages to be removed.
+ * 3) Truncate the pages from the file.
+ *
+ * We are seeing corruptions, if there are crashes in the midst of these
+ * operations, treat all the operations as critical and raise PANIC to
+ * avoid discrepancy between the WAL log, buffer copy and the on-disk image.
*/
- if (RelationNeedsWAL(rel))
+
+ /*
+ * It seems logical and apt to put a critical section, but there are two
+ * issues. First the minor one, palloc() operations down the lane are
+ * restricted, the major one is (as the below comment suggests) for
+ * a repeatable truncate error, WAL replay will never end and the server
+ * is hosed. The former issue can be resolved by either reworking palloc()s
+ * or using TRY/CATCH block instead. The counter argument to the latter
+ * issue is, the lack of critical section is what pushing us into the
+ * scenario we want to avoid by omitting it. Missing critical section is
+ * causing the WAL replay hitting PANIC persistently with invalid-offset
+ * on the disk page ERRORs.
+ */
+ PG_TRY();
{
+
/*
- * Make an XLOG entry reporting the file truncation.
+ * We are going to truncate (shrink) the file, the contents of the
+ * corresponding buffers are useless and need to be discarded, but
+ * a background task might flush them to the disk right after we
+ * truncate and before we discard. Let's prevent it by marking
+ * buffers as IO_IN_PROGRESS (though we don't do any real IO).
*/
- XLogRecPtr lsn;
- xl_smgr_truncate xlrec;
-
- xlrec.blkno = nblocks;
- xlrec.rnode = rel->rd_node;
- xlrec.flags = SMGR_TRUNCATE_ALL;
-
- XLogBeginInsert();
- XLogRegisterData((char *) &xlrec, sizeof(xlrec));
-
- lsn = XLogInsert(RM_SMGR_ID,
- XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE);
+ MarkTruncateBuffers(rel->rd_smgr->smgr_rnode, forks, nforks, blocks);
/*
- * Flush, because otherwise the truncation of the main relation might
- * hit the disk before the WAL record, and the truncation of the FSM
- * or visibility map. If we crashed during that window, we'd be left
- * with a truncated heap, but the FSM or visibility map would still
- * contain entries for the non-existent heap pages.
+ * We WAL-log the truncation before actually truncating, which means
+ * trouble if the truncation fails. If we then crash, the WAL replay
+ * likely isn't going to succeed in the truncation either, and cause a
+ * PANIC. It's tempting to put a critical section here, but that cure
+ * would be worse than the disease. It would turn a usually harmless
+ * failure to truncate, that might spell trouble at WAL replay, into a
+ * certain PANIC.
*/
- if (fsm || vm)
+ if (RelationNeedsWAL(rel))
+ {
+ /*
+ * Make an XLOG entry reporting the file truncation.
+ */
+ XLogRecPtr lsn;
+ xl_smgr_truncate xlrec;
+
+ xlrec.blkno = nblocks;
+ xlrec.rnode = rel->rd_node;
+ xlrec.flags = SMGR_TRUNCATE_ALL;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, sizeof(xlrec));
+
+ lsn = XLogInsert(RM_SMGR_ID,
+ XLOG_SMGR_TRUNCATE | XLR_SPECIAL_REL_UPDATE);
+
+ /*
+ * Flush, because otherwise the truncation of the main relation might
+ * hit the disk before the WAL record, and the truncation of the FSM
+ * or visibility map. If we crashed during that window, we'd be left
+ * with a truncated heap, but the FSM or visibility map would still
+ * contain entries for the non-existent heap pages.
+ */
XLogFlush(lsn);
- }
+ }
- /* Do the real work to truncate relation forks */
- smgrtruncate(rel->rd_smgr, forks, nforks, blocks);
+ MemoryContextAllowInCriticalSection(CurrentMemoryContext, true);
+ /* Do the real work to truncate relation forks */
+ smgrtruncate(rel->rd_smgr, forks, nforks, blocks);
+ MemoryContextAllowInCriticalSection(CurrentMemoryContext, false);
+ }
+ PG_CATCH();
+ {
+ ereport(PANIC, (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("failed to truncate the relation")));
+ }
+ PG_END_TRY();
/*
* Update upper-level FSM pages to account for the truncation.
@@ -689,7 +731,10 @@ smgr_redo(XLogReaderState *record)
/* Do the real work to truncate relation forks */
if (nforks > 0)
+ {
+ MarkTruncateBuffers(reln->smgr_rnode, forks, nforks, blocks);
smgrtruncate(reln, forks, nforks, blocks);
+ }
/*
* Update upper-level FSM pages to account for the truncation.
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e05e2b3456..ad81f873c1 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -141,6 +141,9 @@ static bool IsForInput;
/* local state for LockBufferForCleanup */
static BufferDesc *PinCountWaitBuf = NULL;
+static BufferDesc **truncatedBufs = NULL;
+static int numTruncatedBufs = 0;
+
/*
* Backend-Private refcount management:
*
@@ -3986,6 +3989,9 @@ TerminateBufferIO(BufferDesc *buf, bool clear_dirty, uint32 set_flag_bits)
void
AbortBufferIO(void)
{
+ int i;
+ uint32 buf_state;
+
BufferDesc *buf = InProgressBuf;
if (buf)
@@ -4031,6 +4037,22 @@ AbortBufferIO(void)
}
TerminateBufferIO(buf, false, BM_IO_ERROR);
}
+
+ if (numTruncatedBufs)
+ {
+ Assert(truncatedBufs);
+ for (i = 0; i < numTruncatedBufs; i++)
+ {
+ buf_state = LockBufHdr(truncatedBufs[i]);
+ Assert(buf_state & BM_IO_IN_PROGRESS);
+ buf_state &= ~BM_IO_IN_PROGRESS;
+ UnlockBufHdr(truncatedBufs[i], buf_state);
+ }
+
+ pfree(truncatedBufs);
+ numTruncatedBufs = 0;
+ truncatedBufs = NULL;
+ }
}
/*
@@ -4370,3 +4392,146 @@ TestForOldSnapshot_impl(Snapshot snapshot, Relation relation)
(errcode(ERRCODE_SNAPSHOT_TOO_OLD),
errmsg("snapshot too old")));
}
+
+/*
+ * DropTruncatedBuffers
+ *
+ * This function removes from the buffer pool all the pages from the
+ * saved list of buffers that were marked as BM_IO_IN_PROGRESS just
+ * before the truncation. Dirty pages are simply dropped, without
+ * bothering to write them out first.
+ *
+ * Currently, this is called only from smgr.c smgrtuncate() where the
+ * underlying file was just truncated. It is the responsibility of
+ * MarkTruncatedBuffers() ensure that the list of buffers truncated is
+ * sane and point to the right set of buffers. It is also the responsibility
+ * of higher-level code to ensure that no other process could be trying
+ * to load more pages of the relation into buffers.
+ *
+ */
+void
+DropTruncatedBuffers(RelFileNodeBackend rnode, ForkNumber *forkNum,
+ int nforks, BlockNumber *firstDelBlock)
+{
+ int i;
+ int j;
+
+ /* If it's a local relation, it's localbuf.c's problem. */
+ if (RelFileNodeBackendIsTemp(rnode))
+ {
+ if (rnode.backend == MyBackendId)
+ {
+ for (j = 0; j < nforks; j++)
+ DropRelFileNodeLocalBuffers(rnode.node, forkNum[j],
+ firstDelBlock[j]);
+ }
+ return;
+ }
+
+ if (!numTruncatedBufs)
+ return; /* Nothing to clean up */
+
+ for (i = 0; i < numTruncatedBufs; i++)
+ {
+ uint32 buf_state;
+
+ buf_state = LockBufHdr(truncatedBufs[i]);
+ Assert(buf_state & BM_IO_IN_PROGRESS);
+ InvalidateBuffer(truncatedBufs[i]); /* releases spinlock */
+ }
+
+ pfree(truncatedBufs);
+ numTruncatedBufs = 0;
+ truncatedBufs = NULL;
+}
+
+/*
+ * Finds the buffers (pages) of a about to be truncated-file
+ * and prevent them from being written to disk by marking them
+ * as BM_IO_IN_PROGRESS. Though the buffers are marked for IO, they
+ * will never be written to disk but it prevents processes from
+ * doing IO.
+ *
+ * Note: Buffers might be marked for BM_IO_IN_PROGRES than usual
+ * time because of the extra code path to be exercised before we
+ * drop them, but there shouldn't be any regular backends waiting
+ * on these pages (as they are empty and getting the process of
+ * being discarded).
+ */
+void
+MarkTruncateBuffers(RelFileNodeBackend rnode, ForkNumber *forkNum,
+ int nforks, BlockNumber *firstDelBlock)
+{
+ int i;
+ int j;
+
+ /* Do we care about local buffers i.e. Temp relation ? */
+
+
+ for (i = 0; i < NBuffers; i++)
+ {
+ BufferDesc *bufHdr = GetBufferDescriptor(i);
+ uint32 buf_state;
+
+ /*
+ * Since we take AccessExclusiveLock on the relation during truncate,
+ * it's safe to check without lock and will save lot of lock
+ * acquisitions in typical cases.
+ */
+ if (!RelFileNodeEquals(bufHdr->tag.rnode, rnode.node))
+ continue;
+
+ buf_state = LockBufHdr(bufHdr);
+ for (j = 0; j < nforks; j++)
+ {
+ if (RelFileNodeEquals(bufHdr->tag.rnode, rnode.node) &&
+ bufHdr->tag.forkNum == forkNum[j] &&
+ bufHdr->tag.blockNum >= firstDelBlock[j])
+ {
+retry:
+ if (buf_state & BM_IO_IN_PROGRESS)
+ {
+ UnlockBufHdr(bufHdr, buf_state);
+
+ WaitIO(bufHdr);
+
+ /* OK, now the flag is cleared, recheck */
+ buf_state = LockBufHdr(bufHdr);
+ goto retry;
+ }
+ else
+ {
+ /*
+ * Easy path is to allocate NBuffers, but that
+ * might to lead to wastage, start with 100 and
+ * increase in increments of 100.
+ */
+ if (!truncatedBufs)
+ {
+ truncatedBufs = (BufferDesc **)
+ palloc0(100 * sizeof(BufferDesc *));
+ }
+ else if ((numTruncatedBufs % 100) == 0)
+ {
+ truncatedBufs = (BufferDesc **)
+ repalloc(truncatedBufs,
+ (numTruncatedBufs + 100) * sizeof(BufferDesc *));
+ }
+
+ /*
+ * Add to the list, this will be used either to
+ * clean up (in an exception) or invalidate the
+ * buffers after the truncate.
+ */
+ truncatedBufs[numTruncatedBufs] = bufHdr;
+ numTruncatedBufs++;
+ buf_state |= BM_IO_IN_PROGRESS;
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+ }
+ }
+
+ if (j >= nforks)
+ UnlockBufHdr(bufHdr, buf_state);
+ }
+}
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 360b5bf5bf..d1f066a220 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -572,12 +572,6 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
{
int i;
- /*
- * Get rid of any buffers for the about-to-be-deleted blocks. bufmgr will
- * just drop them without bothering to write the contents.
- */
- DropRelFileNodeBuffers(reln->smgr_rnode, forknum, nforks, nblocks);
-
/*
* Send a shared-inval message to force other backends to close any smgr
* references they may have for this rel. This is useful because they
@@ -608,6 +602,13 @@ smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nb
if (forknum[i] == VISIBILITYMAP_FORKNUM)
reln->smgr_vm_nblocks = nblocks[i];
}
+
+ /*
+ * Now that we have successfully truncated the table(shrunk the file),
+ * get rid of any buffers for the just deleted blocks. Bufmgr will
+ * just drop them without bothering to write the contents.
+ */
+ DropTruncatedBuffers(reln->smgr_rnode, forknum, nforks, nblocks);
}
/*
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index d2a5b52f6e..9889193bd9 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -189,6 +189,10 @@ extern void FlushRelationBuffers(Relation rel);
extern void FlushDatabaseBuffers(Oid dbid);
extern void DropRelFileNodeBuffers(RelFileNodeBackend rnode, ForkNumber *forkNum,
int nforks, BlockNumber *firstDelBlock);
+extern void DropTruncatedBuffers(RelFileNodeBackend rnode, ForkNumber *forkNum,
+ int nforks, BlockNumber *firstDelBlock);
+extern void MarkTruncateBuffers(RelFileNodeBackend rnode, ForkNumber *forkNum,
+ int nforks, BlockNumber *firstDelBlock);
extern void DropRelFileNodesAllBuffers(RelFileNodeBackend *rnodes, int nnodes);
extern void DropDatabaseBuffers(Oid dbid);
--
2.17.1
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-11 00:49 Alvaro Herrera <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 1 reply; 94+ messages in thread
From: Alvaro Herrera @ 2020-04-11 00:49 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; pgsql-hackers; [email protected]
On 2020-Mar-30, Andres Freund wrote:
> If we are really concerned with truncation failing - I don't know why we
> would be, we accept that we have to be able to modify files etc to stay
> up - we can add a pre-check ensuring that permissions are set up
> appropriately to allow us to truncate.
I remember I saw a case where the datadir was NFS or some other network
filesystem thingy, and it lost connection just before autovacuum
truncation, or something like that -- so there was no permission
failure, but the truncate failed and yet PG soldiered on. I think the
connection was re-established soon thereafter and things went back to
"normal", with nobody realizing that a truncate had been lost.
Corruption was discovered a long time afterwards IIRC (weeks or months,
I don't remember).
I didn't review Teja's patch carefully, but the idea of panicking on
failure (causing WAL replay) seems better than the current behavior.
I'd rather put the server to wait until storage is really back.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-11 00:54 Andres Freund <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Andres Freund @ 2020-04-11 00:54 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; pgsql-hackers; [email protected]
Hi,
On 2020-04-10 20:49:05 -0400, Alvaro Herrera wrote:
> On 2020-Mar-30, Andres Freund wrote:
>
> > If we are really concerned with truncation failing - I don't know why we
> > would be, we accept that we have to be able to modify files etc to stay
> > up - we can add a pre-check ensuring that permissions are set up
> > appropriately to allow us to truncate.
>
> I remember I saw a case where the datadir was NFS or some other network
> filesystem thingy, and it lost connection just before autovacuum
> truncation, or something like that -- so there was no permission
> failure, but the truncate failed and yet PG soldiered on. I think the
> connection was re-established soon thereafter and things went back to
> "normal", with nobody realizing that a truncate had been lost.
> Corruption was discovered a long time afterwards IIRC (weeks or months,
> I don't remember).
Yea. In that case we're in a really bad state. Because we truncate after
throwing away the old buffer contents (even if dirty), we'll later read
page contents "from the past". Which won't end well...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-13 06:24 Masahiko Sawada <[email protected]>
parent: Teja Mupparti <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Masahiko Sawada @ 2020-04-13 06:24 UTC (permalink / raw)
To: Teja Mupparti <[email protected]>; +Cc: Andres Freund <[email protected]>; Kyotaro Horiguchi <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On Sat, 11 Apr 2020 at 09:00, Teja Mupparti <[email protected]> wrote:
>
> Thanks Andres and Kyotaro for the quick review. I have fixed the typos and also included the critical section (emulated it with try-catch block since palloc()s are causing issues in the truncate code). This time I used git format-patch.
>
I briefly looked at the latest patch but I'm not sure it's the right
thing here to use PG_TRY/PG_CATCH to report the PANIC error. For
example, with the following code you changed, we will always end up
with emitting a PANIC "failed to truncate the relation" regardless of
the actual cause of the error.
+ PG_CATCH();
+ {
+ ereport(PANIC, (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("failed to truncate the relation")));
+ }
+ PG_END_TRY();
And the comments of RelationTruncate() mentions:
/*
* We WAL-log the truncation before actually truncating, which means
* trouble if the truncation fails. If we then crash, the WAL replay
* likely isn't going to succeed in the truncation either, and cause a
* PANIC. It's tempting to put a critical section here, but that cure
* would be worse than the disease. It would turn a usually harmless
* failure to truncate, that might spell trouble at WAL replay, into a
* certain PANIC.
*/
As a second idea, I wonder if we can defer truncation until commit
time like smgrDoPendingDeletes mechanism. The sequence would be:
At RelationTruncate(),
1. WAL logging.
2. Remember buffers to be dropped.
At CommitTransaction(),
3. Revisit the remembered buffers to check if the buffer still has
table data that needs to be truncated.
4-a, If it has, we mark it as IO_IN_PROGRESS.
4-b, If it already has different table data, ignore it.
5, Truncate physical files.
6, Mark the buffer we marked at #4-a as invalid.
If an error occurs between #3 and #6 or in abort case, we revert all
IO_IN_PROGRESS flags on the buffers.
In the above idea, remembering all buffers having to-be-truncated
table at RelationTruncate(), we reduce the time for checking buffers
at the commit time. Since we acquire AccessExclusiveLock the number of
buffers having to-be-truncated table's data never increases. A
downside would be that since we can truncate multiple relations we
need to remember all buffers of each truncated relations, which is up
to (sizeof(int) * NBuffers) in total.
Regards,
--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-13 08:40 Andres Freund <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Andres Freund @ 2020-04-13 08:40 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
Hi,
On 2020-04-13 15:24:55 +0900, Masahiko Sawada wrote:
> On Sat, 11 Apr 2020 at 09:00, Teja Mupparti <[email protected]> wrote:
> >
> > Thanks Andres and Kyotaro for the quick review. I have fixed the typos and also included the critical section (emulated it with try-catch block since palloc()s are causing issues in the truncate code). This time I used git format-patch.
> >
>
> I briefly looked at the latest patch but I'm not sure it's the right
> thing here to use PG_TRY/PG_CATCH to report the PANIC error. For
> example, with the following code you changed, we will always end up
> with emitting a PANIC "failed to truncate the relation" regardless of
> the actual cause of the error.
>
> + PG_CATCH();
> + {
> + ereport(PANIC, (errcode(ERRCODE_INTERNAL_ERROR),
> + errmsg("failed to truncate the relation")));
> + }
> + PG_END_TRY();
>
> And the comments of RelationTruncate() mentions:
I think that's just a workaround for mdtruncate not being usable in
critical sections.
> /*
> * We WAL-log the truncation before actually truncating, which means
> * trouble if the truncation fails. If we then crash, the WAL replay
> * likely isn't going to succeed in the truncation either, and cause a
> * PANIC. It's tempting to put a critical section here, but that cure
> * would be worse than the disease. It would turn a usually harmless
> * failure to truncate, that might spell trouble at WAL replay, into a
> * certain PANIC.
> */
Yea, but that reasoning is just plain *wrong*. It's *never* ok to WAL
log something and then not perform the action. This leads to to primary
/ replica getting out of sync, crash recovery potentially not completing
(because of records referencing the should-be-truncated pages), ...
> As a second idea, I wonder if we can defer truncation until commit
> time like smgrDoPendingDeletes mechanism. The sequence would be:
This is mostly an issue during [auto]vacuum partially truncating the end
of the file. We intentionally release the AEL regularly to allow other
accesses to continue.
For transactional truncations we don't go down this path (as we create a
new relfilenode).
> At RelationTruncate(),
> 1. WAL logging.
> 2. Remember buffers to be dropped.
You definitely cannot do that, as explained above.
> At CommitTransaction(),
> 3. Revisit the remembered buffers to check if the buffer still has
> table data that needs to be truncated.
> 4-a, If it has, we mark it as IO_IN_PROGRESS.
> 4-b, If it already has different table data, ignore it.
> 5, Truncate physical files.
> 6, Mark the buffer we marked at #4-a as invalid.
>
> If an error occurs between #3 and #6 or in abort case, we revert all
> IO_IN_PROGRESS flags on the buffers.
What would this help with? If we still need the more complicated
truncation sequence?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-13 09:53 Masahiko Sawada <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Masahiko Sawada @ 2020-04-13 09:53 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On Mon, 13 Apr 2020 at 17:40, Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2020-04-13 15:24:55 +0900, Masahiko Sawada wrote:
> > On Sat, 11 Apr 2020 at 09:00, Teja Mupparti <[email protected]> wrote:
> > >
> > > Thanks Andres and Kyotaro for the quick review. I have fixed the typos and also included the critical section (emulated it with try-catch block since palloc()s are causing issues in the truncate code). This time I used git format-patch.
> > >
> >
> > I briefly looked at the latest patch but I'm not sure it's the right
> > thing here to use PG_TRY/PG_CATCH to report the PANIC error. For
> > example, with the following code you changed, we will always end up
> > with emitting a PANIC "failed to truncate the relation" regardless of
> > the actual cause of the error.
> >
> > + PG_CATCH();
> > + {
> > + ereport(PANIC, (errcode(ERRCODE_INTERNAL_ERROR),
> > + errmsg("failed to truncate the relation")));
> > + }
> > + PG_END_TRY();
> >
> > And the comments of RelationTruncate() mentions:
>
> I think that's just a workaround for mdtruncate not being usable in
> critical sections.
>
>
> > /*
> > * We WAL-log the truncation before actually truncating, which means
> > * trouble if the truncation fails. If we then crash, the WAL replay
> > * likely isn't going to succeed in the truncation either, and cause a
> > * PANIC. It's tempting to put a critical section here, but that cure
> > * would be worse than the disease. It would turn a usually harmless
> > * failure to truncate, that might spell trouble at WAL replay, into a
> > * certain PANIC.
> > */
>
> Yea, but that reasoning is just plain *wrong*. It's *never* ok to WAL
> log something and then not perform the action. This leads to to primary
> / replica getting out of sync, crash recovery potentially not completing
> (because of records referencing the should-be-truncated pages), ...
>
>
> > As a second idea, I wonder if we can defer truncation until commit
> > time like smgrDoPendingDeletes mechanism. The sequence would be:
>
> This is mostly an issue during [auto]vacuum partially truncating the end
> of the file. We intentionally release the AEL regularly to allow other
> accesses to continue.
>
> For transactional truncations we don't go down this path (as we create a
> new relfilenode).
>
>
> > At RelationTruncate(),
> > 1. WAL logging.
> > 2. Remember buffers to be dropped.
>
> You definitely cannot do that, as explained above.
Ah yes, you're right.
So it seems to me currently what we can do for this issue would be to
enclose the truncation operation in a critical section. IIUC it's not
enough just to reverse the order of dropping buffers and physical file
truncation because it cannot solve the problem of inconsistency on the
standby. And as Horiguchi-san mentioned, there is no need to reverse
that order if we envelop the truncation operation by a critical
section because we can recover page changes during crash recovery. The
strategy of writing out all dirty buffers before dropping buffers,
proposed as (a) in [1], also seems not enough.
Regards,
[1] https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.deDoing
sync before truncation
--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-14 02:35 Kyotaro Horiguchi <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Kyotaro Horiguchi @ 2020-04-14 02:35 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; pgsql-hackers; [email protected]
At Mon, 13 Apr 2020 18:53:26 +0900, Masahiko Sawada <[email protected]> wrote in
> On Mon, 13 Apr 2020 at 17:40, Andres Freund <[email protected]> wrote:
> >
> > Hi,
> >
> > On 2020-04-13 15:24:55 +0900, Masahiko Sawada wrote:
> > > On Sat, 11 Apr 2020 at 09:00, Teja Mupparti <[email protected]> wrote:
> > > /*
> > > * We WAL-log the truncation before actually truncating, which means
> > > * trouble if the truncation fails. If we then crash, the WAL replay
> > > * likely isn't going to succeed in the truncation either, and cause a
> > > * PANIC. It's tempting to put a critical section here, but that cure
> > > * would be worse than the disease. It would turn a usually harmless
> > > * failure to truncate, that might spell trouble at WAL replay, into a
> > > * certain PANIC.
> > > */
> >
> > Yea, but that reasoning is just plain *wrong*. It's *never* ok to WAL
> > log something and then not perform the action. This leads to to primary
> > / replica getting out of sync, crash recovery potentially not completing
> > (because of records referencing the should-be-truncated pages), ...
It is introduced in 2008 by 3396000684, for 8.4. So it can be said as
an overlook when introducing log-shipping.
The reason other operations like INSERTs (that extends the underlying
file) are "safe" after an extension failure is the following
operations are performed in shared buffers as if the new page exists,
then tries to extend the file again. So if we continue working after
truncation failure, we need to disguise on shared buffers as if the
truncated pages are gone. But we don't have a room for another flag
in buffer header. For example, BM_DIRTY && !BM_VALID might be able to
be used as the state that the page should have been truncated but not
succeeded yet, but I'm not sure.
Anyway, I think the prognosis of a truncation failure is far hopeless
than extension failure in most cases and I doubt that it's good to
introduce such a complex feature only to overcome such a hopeless
situation.
In short, I think we should PANIC in that case.
> > > As a second idea, I wonder if we can defer truncation until commit
> > > time like smgrDoPendingDeletes mechanism. The sequence would be:
> >
> > This is mostly an issue during [auto]vacuum partially truncating the end
> > of the file. We intentionally release the AEL regularly to allow other
> > accesses to continue.
> >
> > For transactional truncations we don't go down this path (as we create a
> > new relfilenode).
> >
> >
> > > At RelationTruncate(),
> > > 1. WAL logging.
> > > 2. Remember buffers to be dropped.
> >
> > You definitely cannot do that, as explained above.
>
> Ah yes, you're right.
>
> So it seems to me currently what we can do for this issue would be to
> enclose the truncation operation in a critical section. IIUC it's not
> enough just to reverse the order of dropping buffers and physical file
> truncation because it cannot solve the problem of inconsistency on the
> standby. And as Horiguchi-san mentioned, there is no need to reverse
> that order if we envelop the truncation operation by a critical
> section because we can recover page changes during crash recovery. The
> strategy of writing out all dirty buffers before dropping buffers,
> proposed as (a) in [1], also seems not enough.
Agreed. Since it's not acceptable ether WAL-logging->not-performing
nor performing->WAL-logging, there's no other way than working as if
truncation is succeeded (and try again) even if it actually
failed. But it would be too complex.
Just making it a critical section seems the right thing here.
> [1] https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
> Doing sync before truncation
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-04-14 19:04 Teja Mupparti <[email protected]>
parent: Kyotaro Horiguchi <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Teja Mupparti @ 2020-04-14 19:04 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; +Cc: [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
Thanks Kyotaro and Masahiko for the feedback. I think there is a consensus on the critical-section around truncate, but I just want to emphasize the need for reversing the order of the dropping the buffers and the truncation.
Repro details (when full page write = off)
1) Page on disk has empty LP 1, Insert into page LP 1
2) checkpoint START (Recovery REDO eventually starts here)
3) Delete all rows on the page (page is empty now)
4) Autovacuum kicks in and truncates the pages
DropRelFileNodeBuffers - Dirty page NOT written, LP 1 on disk still empty
5) Checkpoint completes
6) Crash
7) smgrtruncate - Not reached (this is where we do the physical truncate)
Now the crash-recovery starts
Delete-log-replay (above step-3) reads page with empty LP 1 and the delete fails with PANIC (old page on disk with no insert)
Doing recovery, truncate is even not reached, a WAL replay of the truncation will happen in the future but the recovery fails (repeatedly) even before reaching that point.
Best regards,
Teja
________________________________
From: Kyotaro Horiguchi <[email protected]>
Sent: Monday, April 13, 2020 7:35 PM
To: [email protected] <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: Corruption during WAL replay
At Mon, 13 Apr 2020 18:53:26 +0900, Masahiko Sawada <[email protected]> wrote in
> On Mon, 13 Apr 2020 at 17:40, Andres Freund <[email protected]> wrote:
> >
> > Hi,
> >
> > On 2020-04-13 15:24:55 +0900, Masahiko Sawada wrote:
> > > On Sat, 11 Apr 2020 at 09:00, Teja Mupparti <[email protected]> wrote:
> > > /*
> > > * We WAL-log the truncation before actually truncating, which means
> > > * trouble if the truncation fails. If we then crash, the WAL replay
> > > * likely isn't going to succeed in the truncation either, and cause a
> > > * PANIC. It's tempting to put a critical section here, but that cure
> > > * would be worse than the disease. It would turn a usually harmless
> > > * failure to truncate, that might spell trouble at WAL replay, into a
> > > * certain PANIC.
> > > */
> >
> > Yea, but that reasoning is just plain *wrong*. It's *never* ok to WAL
> > log something and then not perform the action. This leads to to primary
> > / replica getting out of sync, crash recovery potentially not completing
> > (because of records referencing the should-be-truncated pages), ...
It is introduced in 2008 by 3396000684, for 8.4. So it can be said as
an overlook when introducing log-shipping.
The reason other operations like INSERTs (that extends the underlying
file) are "safe" after an extension failure is the following
operations are performed in shared buffers as if the new page exists,
then tries to extend the file again. So if we continue working after
truncation failure, we need to disguise on shared buffers as if the
truncated pages are gone. But we don't have a room for another flag
in buffer header. For example, BM_DIRTY && !BM_VALID might be able to
be used as the state that the page should have been truncated but not
succeeded yet, but I'm not sure.
Anyway, I think the prognosis of a truncation failure is far hopeless
than extension failure in most cases and I doubt that it's good to
introduce such a complex feature only to overcome such a hopeless
situation.
In short, I think we should PANIC in that case.
> > > As a second idea, I wonder if we can defer truncation until commit
> > > time like smgrDoPendingDeletes mechanism. The sequence would be:
> >
> > This is mostly an issue during [auto]vacuum partially truncating the end
> > of the file. We intentionally release the AEL regularly to allow other
> > accesses to continue.
> >
> > For transactional truncations we don't go down this path (as we create a
> > new relfilenode).
> >
> >
> > > At RelationTruncate(),
> > > 1. WAL logging.
> > > 2. Remember buffers to be dropped.
> >
> > You definitely cannot do that, as explained above.
>
> Ah yes, you're right.
>
> So it seems to me currently what we can do for this issue would be to
> enclose the truncation operation in a critical section. IIUC it's not
> enough just to reverse the order of dropping buffers and physical file
> truncation because it cannot solve the problem of inconsistency on the
> standby. And as Horiguchi-san mentioned, there is no need to reverse
> that order if we envelop the truncation operation by a critical
> section because we can recover page changes during crash recovery. The
> strategy of writing out all dirty buffers before dropping buffers,
> proposed as (a) in [1], also seems not enough.
Agreed. Since it's not acceptable ether WAL-logging->not-performing
nor performing->WAL-logging, there's no other way than working as if
truncation is succeeded (and try again) even if it actually
failed. But it would be too complex.
Just making it a critical section seems the right thing here.
> [1] https://www.postgresql.org/message-id/20191207001232.klidxnm756wqxvwx%40alap3.anarazel.de
> Doing sync before truncation
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-06-12 08:20 Masahiko Sawada <[email protected]>
parent: Teja Mupparti <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Masahiko Sawada @ 2020-06-12 08:20 UTC (permalink / raw)
To: Teja Mupparti <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On Wed, 15 Apr 2020 at 04:04, Teja Mupparti <[email protected]> wrote:
>
> Thanks Kyotaro and Masahiko for the feedback. I think there is a consensus on the critical-section around truncate, but I just want to emphasize the need for reversing the order of the dropping the buffers and the truncation.
>
> Repro details (when full page write = off)
>
> 1) Page on disk has empty LP 1, Insert into page LP 1
> 2) checkpoint START (Recovery REDO eventually starts here)
> 3) Delete all rows on the page (page is empty now)
> 4) Autovacuum kicks in and truncates the pages
> DropRelFileNodeBuffers - Dirty page NOT written, LP 1 on disk still empty
> 5) Checkpoint completes
> 6) Crash
> 7) smgrtruncate - Not reached (this is where we do the physical truncate)
>
> Now the crash-recovery starts
>
> Delete-log-replay (above step-3) reads page with empty LP 1 and the delete fails with PANIC (old page on disk with no insert)
>
I agree that when replaying the deletion of (3) the page LP 1 is
empty, but does that replay really fail with PANIC? I guess that we
record that page into invalid_page_tab but don't raise a PANIC in this
case.
Regards,
--
Masahiko Sawada http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-08-17 11:05 Heikki Linnakangas <[email protected]>
parent: Teja Mupparti <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Heikki Linnakangas @ 2020-08-17 11:05 UTC (permalink / raw)
To: Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; +Cc: [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On 14/04/2020 22:04, Teja Mupparti wrote:
> Thanks Kyotaro and Masahiko for the feedback. I think there is a
> consensus on the critical-section around truncate,
+1
> but I just want to emphasize the need for reversing the order of the
> dropping the buffers and the truncation.
>
> Repro details (when full page write = off)
>
> 1) Page on disk has empty LP 1, Insert into page LP 1
> 2) checkpoint START (Recovery REDO eventually starts here)
> 3) Delete all rows on the page (page is empty now)
> 4) Autovacuum kicks in and truncates the pages
> DropRelFileNodeBuffers - Dirty page NOT written, LP 1
> on disk still empty
> 5) Checkpoint completes
> 6) Crash
> 7) smgrtruncate - Not reached (this is where we do the
> physical truncate)
>
> Now the crash-recovery starts
>
> Delete-log-replay (above step-3) reads page with empty LP 1
> and the delete fails with PANIC (old page on disk with no insert)
>
> Doing recovery, truncate is even not reached, a WAL replay of the
> truncation will happen in the future but the recovery fails (repeatedly)
> even before reaching that point.
Hmm. I think simply reversing the order of DropRelFileNodeBuffers() and
truncating the file would open a different issue:
1) Page on disk has empty LP 1, Insert into page LP 1
2) checkpoint START (Recovery REDO eventually starts here)
3) Delete all rows on the page (page is empty now)
4) Autovacuum kicks in and starts truncating
5) smgrtruncate() truncates the file
6) checkpoint writes out buffers for pages that were just truncated
away, expanding the file again.
Your patch had a mechanism to mark the buffers as io-in-progress before
truncating the file to fix that, but I'm wary of that approach. Firstly,
it requires scanning the buffers that are dropped twice, which can take
a long time. I remember that people have already complained that
DropRelFileNodeBuffers() is slow, when it has to scan all the buffers
once. More importantly, abusing the BM_IO_INPROGRESS flag for this seems
bad. For starters, because you're not holding buffer's I/O lock, I
believe the checkpointer would busy-wait on the buffers until the
truncation has completed. See StartBufferIO() and AbortBufferIO().
Perhaps a better approach would be to prevent the checkpoint from
completing, until all in-progress truncations have completed. We have a
mechanism to wait out in-progress commits at the beginning of a
checkpoint, right after the redo point has been established. See
comments around the GetVirtualXIDsDelayingChkpt() function call in
CreateCheckPoint(). We could have a similar mechanism to wait out the
truncations before *completing* a checkpoint.
- Heikki
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-08-17 18:22 Andres Freund <[email protected]>
parent: Heikki Linnakangas <[email protected]>
0 siblings, 3 replies; 94+ messages in thread
From: Andres Freund @ 2020-08-17 18:22 UTC (permalink / raw)
To: Heikki Linnakangas <[email protected]>; +Cc: Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
Hi,
On 2020-08-17 14:05:37 +0300, Heikki Linnakangas wrote:
> On 14/04/2020 22:04, Teja Mupparti wrote:
> > Thanks Kyotaro and Masahiko for the feedback. I think there is a
> > consensus on the critical-section around truncate,
>
> +1
I'm inclined to think that we should do that independent of the far more
complicated fix for other related issues.
> > but I just want to emphasize the need for reversing the order of the
> > dropping the buffers and the truncation.
> >
> > Repro details (when full page write = off)
> >
> > 1) Page on disk has empty LP 1, Insert into page LP 1
> > 2) checkpoint START (Recovery REDO eventually starts here)
> > 3) Delete all rows on the page (page is empty now)
> > 4) Autovacuum kicks in and truncates the pages
> > DropRelFileNodeBuffers - Dirty page NOT written, LP 1
> > on disk still empty
> > 5) Checkpoint completes
> > 6) Crash
> > 7) smgrtruncate - Not reached (this is where we do the
> > physical truncate)
> >
> > Now the crash-recovery starts
> >
> > Delete-log-replay (above step-3) reads page with empty LP 1
> > and the delete fails with PANIC (old page on disk with no insert)
> >
> > Doing recovery, truncate is even not reached, a WAL replay of the
> > truncation will happen in the future but the recovery fails (repeatedly)
> > even before reaching that point.
>
> Hmm. I think simply reversing the order of DropRelFileNodeBuffers() and
> truncating the file would open a different issue:
>
> 1) Page on disk has empty LP 1, Insert into page LP 1
> 2) checkpoint START (Recovery REDO eventually starts here)
> 3) Delete all rows on the page (page is empty now)
> 4) Autovacuum kicks in and starts truncating
> 5) smgrtruncate() truncates the file
> 6) checkpoint writes out buffers for pages that were just truncated away,
> expanding the file again.
>
> Your patch had a mechanism to mark the buffers as io-in-progress before
> truncating the file to fix that, but I'm wary of that approach. Firstly, it
> requires scanning the buffers that are dropped twice, which can take a long
> time.
I was thinking that we'd keep track of all the buffers marked as "in
progress" that way, avoiding the second scan.
It's also worth keeping in mind that this code is really only relevant
for partial truncations, which don't happen at the same frequency as
transactional truncations.
> I remember that people have already complained that
> DropRelFileNodeBuffers() is slow, when it has to scan all the buffers
> once.
But that's when dropping many relations, normally. E.g. at the end of a
regression test.
> More importantly, abusing the BM_IO_INPROGRESS flag for this seems
> bad. For starters, because you're not holding buffer's I/O lock, I
> believe the checkpointer would busy-wait on the buffers until the
> truncation has completed. See StartBufferIO() and AbortBufferIO().
I think we should apply Robert's patch that makes io locks into
condition variables. Then we can fairly easily have many many buffers io
locked. Obviously there's some issues with doing so in the back
branches :(
I'm working on an AIO branch, and that also requires to be able to mark
multiple buffers as in-progress, FWIW.
> Perhaps a better approach would be to prevent the checkpoint from
> completing, until all in-progress truncations have completed. We have a
> mechanism to wait out in-progress commits at the beginning of a checkpoint,
> right after the redo point has been established. See comments around the
> GetVirtualXIDsDelayingChkpt() function call in CreateCheckPoint(). We could
> have a similar mechanism to wait out the truncations before *completing* a
> checkpoint.
What I outlined earlier *is* essentially a way to do so, by preventing
checkpointing from finishing the buffer scan while a dangerous state
exists.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-10-30 16:34 Anastasia Lubennikova <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 0 replies; 94+ messages in thread
From: Anastasia Lubennikova @ 2020-10-30 16:34 UTC (permalink / raw)
To: [email protected]; +Cc: Teja Mupparti <[email protected]>
Status update for a commitfest entry.
I see quite a few unanswered questions in the thread since the last patch version was sent. So, I move it to "Waiting on Author".
The new status of this patch is: Waiting on Author
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-11-06 11:40 Masahiko Sawada <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 1 reply; 94+ messages in thread
From: Masahiko Sawada @ 2020-11-06 11:40 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On Tue, Aug 18, 2020 at 3:22 AM Andres Freund <[email protected]> wrote:
>
> Hi,
>
> On 2020-08-17 14:05:37 +0300, Heikki Linnakangas wrote:
> > On 14/04/2020 22:04, Teja Mupparti wrote:
> > > Thanks Kyotaro and Masahiko for the feedback. I think there is a
> > > consensus on the critical-section around truncate,
> >
> > +1
>
> I'm inclined to think that we should do that independent of the far more
> complicated fix for other related issues.
+1
If we had a critical section in RelationTruncate(), crash recovery
would continue failing until the situation of the underlying file is
recovered if a PANIC happens. The current comment in
RelationTruncate() says it’s worse than the disease. But considering
physical replication, as Andres mentioned, a failure to truncate the
file after logging WAL is no longer a harmless failure. Also, the
critical section would be necessary even if we reversed the order of
truncation and dropping buffers and resolved the issue. So I agree to
proceed with the patch that adds a critical section independent of
fixing other related things discussed in this thread. If Teja seems
not to work on this I’ll write the patch.
Regards,
--
Masahiko Sawada
EnterpriseDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2020-12-01 14:58 Anastasia Lubennikova <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Anastasia Lubennikova @ 2020-12-01 14:58 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Andres Freund <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Teja Mupparti <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected] <[email protected]>; pgsql-hackers; [email protected] <[email protected]>
On 06.11.2020 14:40, Masahiko Sawada wrote:
>
> So I agree to
> proceed with the patch that adds a critical section independent of
> fixing other related things discussed in this thread. If Teja seems
> not to work on this I’ll write the patch.
>
> Regards,
>
>
> --
> Masahiko Sawada
> EnterpriseDB: https://www.enterprisedb.com/
>
>
Status update for a commitfest entry.
The commitfest is closed now. As this entry is a bug fix, I am moving it
to the next CF.
Are you planning to continue working on it?
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH 4/5] Change conversion function signature.
@ 2020-12-16 10:13 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2020-12-16 10:13 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
TODO: Upgrade?
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 13 +-
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
33 files changed, 1292 insertions(+), 633 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index 0ee3b6d19a3..2d032a802f5 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 3558e660c73..8b9d794dd33 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2266,6 +2266,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index 192948caad2..91251cf70e7 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 376b48ca611..986c0c0c37d 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 3628e690aa1..40f231c12dc 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index 59c6c3bb129..ad9ebac39b1 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index ea05436596d..81064cb6e98 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index 600c5cbc5cd..5a44262834a 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 7794d7ef8bf..bf87335c6a0 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,23 +206,30 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -213,8 +244,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -238,8 +273,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -257,15 +296,18 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -276,8 +318,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -306,22 +352,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -332,8 +385,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -341,8 +398,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -364,20 +425,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -391,8 +459,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -400,8 +472,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -414,20 +490,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -440,8 +523,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -449,8 +536,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -464,16 +555,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index f424f881459..8752dcc09ac 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index a358a707c11..431971c40cb 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index 75ed49ac54e..e45c7718945 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index 90ad316111a..e8303f38b68 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index 018312489cb..d2d9c44b3ff 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index 62182a9ba8b..9892db0d102 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index dc5abb5dfd4..88ea32b74ba 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 088a38d8390..11dee117f47 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index a9fe94f88b8..29c03512819 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index 96909b58859..72677fa6d40 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 78bbcd3ce7d..057bc65e521 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 348524f4a2c..d16b6fe31d8 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 2cdca9f780d..0bc08829657 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index e09a7c8e41e..a760ab54ab6 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index c56fa80a4bb..23892790730 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 458500998d4..94930659347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index 3226ed03258..dfdc0dbfa2f 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 1a0074d063c..8f046280029 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 67d1c4fc19f..a585e3a6f1e 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e6c7b070f64..1dd5558b078 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10409,388 +10409,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 549f2dd045d..4529a15a6ba 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 507b474b1bb..e4aab19ddaa 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1035,13 +1035,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index 4189a5a4e09..90a735ea5c8 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -542,13 +542,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.20.1
--------------28FF91F97438A3BA06B3DA61
Content-Type: text/x-patch; charset=UTF-8;
name="0005-Do-COPY-FROM-encoding-conversion-verification-in-lar.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="0005-Do-COPY-FROM-encoding-conversion-verification-in-lar.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-01-06 08:33 Kyotaro Horiguchi <[email protected]>
parent: Andres Freund <[email protected]>
2 siblings, 1 reply; 94+ messages in thread
From: Kyotaro Horiguchi @ 2021-01-06 08:33 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]
At Mon, 17 Aug 2020 11:22:15 -0700, Andres Freund <[email protected]> wrote in
> Hi,
>
> On 2020-08-17 14:05:37 +0300, Heikki Linnakangas wrote:
> > On 14/04/2020 22:04, Teja Mupparti wrote:
> > > Thanks Kyotaro and Masahiko for the feedback. I think there is a
> > > consensus on the critical-section around truncate,
> >
> > +1
>
> I'm inclined to think that we should do that independent of the far more
> complicated fix for other related issues.
...
> > Perhaps a better approach would be to prevent the checkpoint from
> > completing, until all in-progress truncations have completed. We have a
> > mechanism to wait out in-progress commits at the beginning of a checkpoint,
> > right after the redo point has been established. See comments around the
> > GetVirtualXIDsDelayingChkpt() function call in CreateCheckPoint(). We could
> > have a similar mechanism to wait out the truncations before *completing* a
> > checkpoint.
>
> What I outlined earlier *is* essentially a way to do so, by preventing
> checkpointing from finishing the buffer scan while a dangerous state
> exists.
Seems reasonable. The attached does that. It actually works for the
initial case.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments:
[text/x-patch] delay_chkpt_cmpl_after_trunc_sucess.patch (13.2K, ../../[email protected]/2-delay_chkpt_cmpl_after_trunc_sucess.patch)
download | inline diff:
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1233448481..e1d3068f14 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3062,8 +3062,8 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
* crash/basebackup, even though the state of the data directory would
* require it.
*/
- Assert(!MyProc->delayChkpt);
- MyProc->delayChkpt = true;
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
+ MyProc->delayChkpt = DELAY_CHKPT_START;
/* WAL log truncation */
WriteMTruncateXlogRec(newOldestMultiDB,
@@ -3089,7 +3089,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
/* Then offsets */
PerformOffsetsTruncation(oldestMulti, newOldestMulti);
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
END_CRIT_SECTION();
LWLockRelease(MultiXactTruncationLock);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index fc18b77832..1b74a229d6 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -463,7 +463,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
proc->lxid = (LocalTransactionId) xid;
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
- proc->delayChkpt = false;
+ proc->delayChkpt = DELAY_CHKPT_NONE;
proc->statusFlags = 0;
proc->pid = 0;
proc->backendId = InvalidBackendId;
@@ -1108,7 +1108,8 @@ EndPrepare(GlobalTransaction gxact)
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
+ MyProc->delayChkpt = DELAY_CHKPT_START;
XLogBeginInsert();
for (record = records.head; record != NULL; record = record->next)
@@ -1151,7 +1152,7 @@ EndPrepare(GlobalTransaction gxact)
* checkpoint starting after this will certainly see the gxact as a
* candidate for fsyncing.
*/
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
/*
* Remember that we have this GlobalTransaction entry locked for us. If
@@ -2199,7 +2200,8 @@ RecordTransactionCommitPrepared(TransactionId xid,
START_CRIT_SECTION();
/* See notes in RecordTransactionCommit */
- MyProc->delayChkpt = true;
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
+ MyProc->delayChkpt = DELAY_CHKPT_START;
/*
* Emit the XLOG commit record. Note that we mark 2PC commits as
@@ -2247,7 +2249,7 @@ RecordTransactionCommitPrepared(TransactionId xid,
TransactionIdCommitTree(xid, nchildren, children);
/* Checkpoint can proceed now */
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
END_CRIT_SECTION();
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index a2068e3fd4..000206e506 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1334,8 +1334,9 @@ RecordTransactionCommit(void)
* This makes checkpoint's determination of which xacts are delayChkpt
* a bit fuzzy, but it doesn't matter.
*/
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ MyProc->delayChkpt = DELAY_CHKPT_START;
SetCurrentTransactionStopTimestamp();
@@ -1436,7 +1437,7 @@ RecordTransactionCommit(void)
*/
if (markXidCommitted)
{
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
END_CRIT_SECTION();
}
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index ede93ad7fd..5b0a653408 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9024,18 +9024,30 @@ CreateCheckPoint(int flags)
* and we will correctly flush the update below. So we cannot miss any
* xacts we need to wait for.
*/
- vxids = GetVirtualXIDsDelayingChkpt(&nvxids);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_START);
if (nvxids > 0)
{
do
{
pg_usleep(10000L); /* wait for 10 msec */
- } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids));
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_START));
}
pfree(vxids);
CheckPointGuts(checkPoint.redo, flags);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_COMPLETE);
+ if (nvxids > 0)
+ {
+ do
+ {
+ pg_usleep(10000L); /* wait for 10 msec */
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_COMPLETE));
+ }
+ pfree(vxids);
+
/*
* Take a snapshot of running transactions and write this to WAL. This
* allows us to reconstruct the state of running transactions during
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 7052dc245e..51f2581c06 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -923,7 +923,7 @@ XLogSaveBufferForHint(Buffer buffer, bool buffer_std)
/*
* Ensure no checkpoint can change our view of RedoRecPtr.
*/
- Assert(MyProc->delayChkpt);
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_START);
/*
* Update RedoRecPtr so that we can make the right decision
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index cba7a9ada0..b75ea97b04 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -325,6 +325,16 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
RelationPreTruncate(rel);
+ /*
+ * If the file truncation fails but the concurrent checkpoint completes
+ * just before that, the next crash recovery can fail due to WAL records
+ * inconsistent with the untruncated pages. To avoid that situation we
+ * delay the checkpoint completion until we confirm the truncation to be
+ * successful.
+ */
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
+ MyProc->delayChkpt = DELAY_CHKPT_COMPLETE;
+
/*
* We WAL-log the truncation before actually truncating, which means
* trouble if the truncation fails. If we then crash, the WAL replay
@@ -373,6 +383,8 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
*/
if (need_fsm_vacuum)
FreeSpaceMapVacuumRange(rel, nblocks, InvalidBlockNumber);
+
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
}
/*
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 8f2c482bc8..52d70019c4 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3615,7 +3615,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
{
XLogRecPtr lsn = InvalidXLogRecPtr;
bool dirtied = false;
- bool delayChkpt = false;
+ int delayChkpt = DELAY_CHKPT_NONE;
uint32 buf_state;
/*
@@ -3665,7 +3665,8 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
* essential that CreateCheckpoint waits for virtual transactions
* rather than full transactionids.
*/
- MyProc->delayChkpt = delayChkpt = true;
+ Assert(MyProc->delayChkpt == DELAY_CHKPT_NONE);
+ MyProc->delayChkpt = delayChkpt = DELAY_CHKPT_START;
lsn = XLogSaveBufferForHint(buffer, buffer_std);
}
@@ -3698,7 +3699,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
UnlockBufHdr(bufHdr, buf_state);
if (delayChkpt)
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
if (dirtied)
{
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index cf12eda504..20757274ec 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -655,7 +655,10 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = DELAY_CHKPT_NONE;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -694,7 +697,10 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
proc->xid = InvalidTransactionId;
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = DELAY_CHKPT_NONE;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -2929,7 +2935,8 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* delaying checkpoint because they have critical actions in progress.
*
* Constructs an array of VXIDs of transactions that are currently in commit
- * critical sections, as shown by having delayChkpt set in their PGPROC.
+ * critical sections, as shown by having delayChkpt set to the specified value
+ * in their PGPROC.
*
* Returns a palloc'd array that should be freed by the caller.
* *nvxids is the number of valid entries.
@@ -2943,13 +2950,15 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* for clearing of delayChkpt to propagate is unimportant for correctness.
*/
VirtualTransactionId *
-GetVirtualXIDsDelayingChkpt(int *nvxids)
+GetVirtualXIDsDelayingChkpt(int *nvxids, DelayChkptType type)
{
VirtualTransactionId *vxids;
ProcArrayStruct *arrayP = procArray;
int count = 0;
int index;
+ Assert(type != DELAY_CHKPT_NONE);
+
/* allocate what's certainly enough result space */
vxids = (VirtualTransactionId *)
palloc(sizeof(VirtualTransactionId) * arrayP->maxProcs);
@@ -2961,7 +2970,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];
- if (proc->delayChkpt)
+ if (proc->delayChkpt == type)
{
VirtualTransactionId vxid;
@@ -2987,12 +2996,15 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
* those numbers should be small enough for it not to be a problem.
*/
bool
-HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
+HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids,
+ DelayChkptType type)
{
bool result = false;
ProcArrayStruct *arrayP = procArray;
int index;
+ Assert(type != DELAY_CHKPT_NONE);
+
LWLockAcquire(ProcArrayLock, LW_SHARED);
for (index = 0; index < arrayP->numProcs; index++)
@@ -3003,7 +3015,7 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
GET_VXID_FROM_PGPROC(vxid, *proc);
- if (proc->delayChkpt && VirtualTransactionIdIsValid(vxid))
+ if (proc->delayChkpt == type && VirtualTransactionIdIsValid(vxid))
{
int i;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 9b6aa2fe0d..b7f9310afe 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -392,7 +392,7 @@ InitProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
if (IsAutoVacuumWorkerProcess())
@@ -573,7 +573,7 @@ InitAuxiliaryProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = DELAY_CHKPT_NONE;
MyProc->statusFlags = 0;
MyProc->lwWaiting = false;
MyProc->lwWaitMode = 0;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 989c5849d4..ca764a1a72 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -78,6 +78,14 @@ struct XidCache
*/
#define INVALID_PGPROCNO PG_INT32_MAX
+/* type for PGPROC.delayChkpt */
+typedef enum DelayChkptType
+{
+ DELAY_CHKPT_NONE = 0,
+ DELAY_CHKPT_START,
+ DELAY_CHKPT_COMPLETE
+} DelayChkptType;
+
typedef enum
{
PROC_WAIT_STATUS_OK,
@@ -181,7 +189,8 @@ struct PGPROC
LOCKMASK heldLocks; /* bitmask for lock types already held on this
* lock object by this backend */
- bool delayChkpt; /* true if this proc delays checkpoint start */
+ DelayChkptType delayChkpt; /* if this proc delays checkpoint start and/or
+ * completion. */
uint8 statusFlags; /* this backend's status flags, see PROC_*
* above. mirrored in
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index b01fa52139..e560cada2b 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -15,11 +15,11 @@
#define PROCARRAY_H
#include "storage/lock.h"
+#include "storage/proc.h"
#include "storage/standby.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
-
extern Size ProcArrayShmemSize(void);
extern void CreateSharedProcArray(void);
extern void ProcArrayAdd(PGPROC *proc);
@@ -59,8 +59,10 @@ extern TransactionId GetOldestActiveTransactionId(void);
extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly);
extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin);
-extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids);
-extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids);
+extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids,
+ DelayChkptType type);
+extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
+ int nvxids, DelayChkptType type);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v2 1/2] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 13 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1387 insertions(+), 633 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..b41d3e0bb9a 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b5f52d4e4a3..f97a911a716 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------B7FB7AC96BFBBB314DDCCEF3
Content-Type: text/x-patch; charset=UTF-8;
name="v2-0002-Do-COPY-FROM-encoding-conversion-verification-in-.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v2-0002-Do-COPY-FROM-encoding-conversion-verification-in-.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* [PATCH v3 2/5] Change conversion function signature.
@ 2021-01-28 16:42 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Heikki Linnakangas @ 2021-01-28 16:42 UTC (permalink / raw)
Add a 'noError' argument, so that we can try to convert a buffer without
knowing the character boundaries beforehand. The functions now need to
return the number of input bytes successfully converted.
This is is a backwards-incompatible change, if you have created a custom
encoding conversion with CREATE CONVERSION. This adds a check to
pg_upgrade for that, refusing the upgrade if there are any user-defined
encoding conversions.
---
doc/src/sgml/ref/create_conversion.sgml | 5 +-
src/backend/commands/conversioncmds.c | 27 +-
src/backend/utils/error/elog.c | 2 +
src/backend/utils/mb/conv.c | 112 +++++-
.../cyrillic_and_mic/cyrillic_and_mic.c | 127 ++++---
.../euc2004_sjis2004/euc2004_sjis2004.c | 96 ++++-
.../euc_cn_and_mic/euc_cn_and_mic.c | 57 ++-
.../euc_jp_and_sjis/euc_jp_and_sjis.c | 153 ++++++--
.../euc_kr_and_mic/euc_kr_and_mic.c | 57 ++-
.../euc_tw_and_big5/euc_tw_and_big5.c | 165 +++++++--
.../latin2_and_win1250/latin2_and_win1250.c | 49 ++-
.../latin_and_mic/latin_and_mic.c | 43 ++-
.../utf8_and_big5/utf8_and_big5.c | 37 +-
.../utf8_and_cyrillic/utf8_and_cyrillic.c | 67 ++--
.../utf8_and_euc2004/utf8_and_euc2004.c | 37 +-
.../utf8_and_euc_cn/utf8_and_euc_cn.c | 37 +-
.../utf8_and_euc_jp/utf8_and_euc_jp.c | 37 +-
.../utf8_and_euc_kr/utf8_and_euc_kr.c | 37 +-
.../utf8_and_euc_tw/utf8_and_euc_tw.c | 37 +-
.../utf8_and_gb18030/utf8_and_gb18030.c | 37 +-
.../utf8_and_gbk/utf8_and_gbk.c | 37 +-
.../utf8_and_iso8859/utf8_and_iso8859.c | 43 ++-
.../utf8_and_iso8859_1/utf8_and_iso8859_1.c | 27 +-
.../utf8_and_johab/utf8_and_johab.c | 37 +-
.../utf8_and_sjis/utf8_and_sjis.c | 37 +-
.../utf8_and_sjis2004/utf8_and_sjis2004.c | 37 +-
.../utf8_and_uhc/utf8_and_uhc.c | 37 +-
.../utf8_and_win/utf8_and_win.c | 43 ++-
src/backend/utils/mb/mbutils.c | 18 +-
src/bin/pg_upgrade/check.c | 95 +++++
src/include/catalog/pg_proc.dat | 332 +++++++++---------
src/include/mb/pg_wchar.h | 49 +--
src/test/regress/expected/opr_sanity.out | 7 +-
src/test/regress/sql/opr_sanity.sql | 7 +-
34 files changed, 1390 insertions(+), 635 deletions(-)
diff --git a/doc/src/sgml/ref/create_conversion.sgml b/doc/src/sgml/ref/create_conversion.sgml
index e7700fecfc5..f014a676c88 100644
--- a/doc/src/sgml/ref/create_conversion.sgml
+++ b/doc/src/sgml/ref/create_conversion.sgml
@@ -117,8 +117,9 @@ conv_proc(
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
- integer -- source string length
-) RETURNS void;
+ integer, -- source string length
+ boolean -- if true, don't throw an error if conversion fails
+) RETURNS integer;
</programlisting></para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/conversioncmds.c b/src/backend/commands/conversioncmds.c
index f7ff321de71..d2041466911 100644
--- a/src/backend/commands/conversioncmds.c
+++ b/src/backend/commands/conversioncmds.c
@@ -45,8 +45,9 @@ CreateConversionCommand(CreateConversionStmt *stmt)
const char *from_encoding_name = stmt->for_encoding_name;
const char *to_encoding_name = stmt->to_encoding_name;
List *func_name = stmt->func_name;
- static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID};
+ static const Oid funcargs[] = {INT4OID, INT4OID, CSTRINGOID, INTERNALOID, INT4OID, BOOLOID};
char result[1];
+ Datum funcresult;
/* Convert list of names to a name and namespace */
namespaceId = QualifiedNameGetCreationNamespace(stmt->conversion_name,
@@ -92,8 +93,8 @@ CreateConversionCommand(CreateConversionStmt *stmt)
funcoid = LookupFuncName(func_name, sizeof(funcargs) / sizeof(Oid),
funcargs, false);
- /* Check it returns VOID, else it's probably the wrong function */
- if (get_func_rettype(funcoid) != VOIDOID)
+ /* Check it returns int4, else it's probably the wrong function */
+ if (get_func_rettype(funcoid) != INT4OID)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("encoding conversion function %s must return type %s",
@@ -111,12 +112,20 @@ CreateConversionCommand(CreateConversionStmt *stmt)
* string; the conversion function should throw an error if it can't
* perform the requested conversion.
*/
- OidFunctionCall5(funcoid,
- Int32GetDatum(from_encoding),
- Int32GetDatum(to_encoding),
- CStringGetDatum(""),
- CStringGetDatum(result),
- Int32GetDatum(0));
+ funcresult = OidFunctionCall6(funcoid,
+ Int32GetDatum(from_encoding),
+ Int32GetDatum(to_encoding),
+ CStringGetDatum(""),
+ CStringGetDatum(result),
+ Int32GetDatum(0),
+ BoolGetDatum(false));
+
+ /* The function should return 0 for empty input. Might as well check that, too. */
+ if (DatumGetInt32(funcresult) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("encoding conversion function %s returned incorrect result for empty input",
+ NameListToString(func_name))));
/*
* All seem ok, go ahead (possible failure would be a duplicate conversion
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 80c26724612..762f77d533c 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2280,6 +2280,8 @@ write_console(const char *line, int len)
* Conversion on non-win32 platforms is not implemented yet. It requires
* non-throw version of pg_do_encoding_conversion(), that converts
* unconvertable characters to '?' without errors.
+ *
+ * XXX: We have a no-throw version now. It doesn't convert to '?' though.
*/
#endif
diff --git a/src/backend/utils/mb/conv.c b/src/backend/utils/mb/conv.c
index a07b54bd3b8..b83358bc7a5 100644
--- a/src/backend/utils/mb/conv.c
+++ b/src/backend/utils/mb/conv.c
@@ -26,14 +26,16 @@
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the target charset, or 0 if there is no equivalent code.
*/
-void
+int
local2local(const unsigned char *l,
unsigned char *p,
int len,
int src_encoding,
int dest_encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -41,7 +43,11 @@ local2local(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(src_encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -50,13 +56,19 @@ local2local(const unsigned char *l,
if (c2)
*p++ = c2;
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(src_encoding, dest_encoding,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -67,17 +79,22 @@ local2local(const unsigned char *l,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = l;
int c1;
while (len > 0)
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (IS_HIGHBIT_SET(c1))
*p++ = lc;
*p++ = c1;
@@ -85,6 +102,8 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -95,17 +114,22 @@ latin2mic(const unsigned char *l, unsigned char *p, int len,
* lc is the mule character set id for the local encoding
* encoding is the PG identifier for the local encoding
*/
-void
+int
mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding)
+ int lc, int encoding, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -118,17 +142,27 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]))
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
+ }
*p++ = mic[1];
mic += 2;
len -= 2;
}
}
*p = '\0';
+
+ return mic - start;
}
@@ -144,14 +178,16 @@ mic2latin(const unsigned char *mic, unsigned char *p, int len,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the mule encoding, or 0 if there is no equivalent code.
*/
-void
+int
latin2mic_with_table(const unsigned char *l,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = l;
unsigned char c1,
c2;
@@ -159,7 +195,11 @@ latin2mic_with_table(const unsigned char *l,
{
c1 = *l;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(encoding, (const char *) l, len);
+ }
if (!IS_HIGHBIT_SET(c1))
*p++ = c1;
else
@@ -171,13 +211,19 @@ latin2mic_with_table(const unsigned char *l,
*p++ = c2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_MULE_INTERNAL,
(const char *) l, len);
+ }
}
l++;
len--;
}
*p = '\0';
+
+ return l - start;
}
/*
@@ -192,14 +238,16 @@ latin2mic_with_table(const unsigned char *l,
* starting from 128 (0x80). each entry in the table holds the corresponding
* code point for the local charset, or 0 if there is no equivalent code.
*/
-void
+int
mic2latin_with_table(const unsigned char *mic,
unsigned char *p,
int len,
int lc,
int encoding,
- const unsigned char *tab)
+ const unsigned char *tab,
+ bool noError)
{
+ const unsigned char *start = mic;
unsigned char c1,
c2;
@@ -207,7 +255,11 @@ mic2latin_with_table(const unsigned char *mic,
{
c1 = *mic;
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic, len);
+ }
if (!IS_HIGHBIT_SET(c1))
{
/* easy for ASCII */
@@ -220,11 +272,17 @@ mic2latin_with_table(const unsigned char *mic,
int l = pg_mule_mblen(mic);
if (len < l)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL, (const char *) mic,
len);
+ }
if (l != 2 || c1 != lc || !IS_HIGHBIT_SET(mic[1]) ||
(c2 = tab[mic[1] - HIGHBIT]) == 0)
{
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, encoding,
(const char *) mic, len);
break; /* keep compiler quiet */
@@ -235,6 +293,8 @@ mic2latin_with_table(const unsigned char *mic,
}
}
*p = '\0';
+
+ return mic - start;
}
/*
@@ -425,17 +485,19 @@ pg_mb_radix_conv(const pg_mb_radix_tree *rt,
*
* See pg_wchar.h for more details about the data structures used here.
*/
-void
+int
UtfToLocal(const unsigned char *utf, int len,
unsigned char *iso,
const pg_mb_radix_tree *map,
const pg_utf_to_local_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding, bool noError)
{
uint32 iutf;
int l;
const pg_utf_to_local_combined *cp;
+ const unsigned char *start = utf;
+ const unsigned char *cur = utf;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -449,6 +511,8 @@ UtfToLocal(const unsigned char *utf, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*utf == '\0')
break;
@@ -584,15 +648,19 @@ UtfToLocal(const unsigned char *utf, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, encoding,
(const char *) (utf - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(PG_UTF8, (const char *) utf, len);
*iso = '\0';
+
+ return cur - start;
}
/*
@@ -616,18 +684,24 @@ UtfToLocal(const unsigned char *utf, int len,
* (if provided) is applied. An error is raised if no match is found.
*
* See pg_wchar.h for more details about the data structures used here.
+ *
+ * Returns the number of input bytes consumed. If noError is true, this can
+ * be less than 'len'.
*/
-void
+int
LocalToUtf(const unsigned char *iso, int len,
unsigned char *utf,
const pg_mb_radix_tree *map,
const pg_local_to_utf_combined *cmap, int cmapsize,
utf_local_conversion_func conv_func,
- int encoding)
+ int encoding,
+ bool noError)
{
uint32 iiso;
int l;
const pg_local_to_utf_combined *cp;
+ const unsigned char *start = iso;
+ const unsigned char *cur = iso;
if (!PG_VALID_ENCODING(encoding))
ereport(ERROR,
@@ -641,6 +715,8 @@ LocalToUtf(const unsigned char *iso, int len,
unsigned char b3 = 0;
unsigned char b4 = 0;
+ cur = iso;
+
/* "break" cases all represent errors */
if (*iso == '\0')
break;
@@ -723,13 +799,17 @@ LocalToUtf(const unsigned char *iso, int len,
}
/* failed to translate this character */
+ if (noError)
+ break;
report_untranslatable_char(encoding, PG_UTF8,
(const char *) (iso - l), len);
}
/* if we broke out of loop early, must be invalid input */
- if (len > 0)
+ if (len > 0 && !noError)
report_invalid_encoding(encoding, (const char *) iso, len);
*utf = '\0';
+
+ return cur - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
index 4c5b02654de..368c2deb5e4 100644
--- a/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/cyrillic_and_mic/cyrillic_and_mic.c
@@ -44,8 +44,11 @@ PG_FUNCTION_INFO_V1(win866_to_iso);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -306,12 +309,14 @@ koi8r_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = latin2mic(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -320,12 +325,14 @@ mic_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_KOI8R);
- mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R);
+ converted = mic2latin(src, dest, len, LC_KOI8_R, PG_KOI8R, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -334,12 +341,14 @@ iso_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -348,12 +357,14 @@ mic_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_ISO_8859_5);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -362,12 +373,14 @@ win1251_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -376,12 +389,14 @@ mic_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1251);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -390,12 +405,14 @@ win866_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi);
+ converted = latin2mic_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -404,12 +421,14 @@ mic_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN866);
- mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866);
+ converted = mic2latin_with_table(src, dest, len, LC_KOI8_R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -418,12 +437,14 @@ koi8r_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN1251);
- local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN1251, koi2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -432,12 +453,14 @@ win1251_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_KOI8R);
- local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_KOI8R, win12512koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -446,12 +469,14 @@ koi8r_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_WIN866);
- local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_WIN866, koi2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -460,12 +485,14 @@ win866_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_KOI8R);
- local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi);
+ converted = local2local(src, dest, len, PG_WIN866, PG_KOI8R, win8662koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -474,12 +501,14 @@ win866_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_WIN1251);
- local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251);
+ converted = local2local(src, dest, len, PG_WIN866, PG_WIN1251, win8662win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -488,12 +517,14 @@ win1251_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_WIN866);
- local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_WIN866, win12512win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -502,12 +533,14 @@ iso_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_KOI8R);
- local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_KOI8R, iso2koi, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -516,12 +549,14 @@ koi8r_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_ISO_8859_5);
- local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso);
+ converted = local2local(src, dest, len, PG_KOI8R, PG_ISO_8859_5, koi2iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -530,12 +565,14 @@ iso_to_win1251(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN1251);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN1251, iso2win1251, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -544,12 +581,14 @@ win1251_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1251, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso);
+ converted = local2local(src, dest, len, PG_WIN1251, PG_ISO_8859_5, win12512iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -558,12 +597,14 @@ iso_to_win866(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_ISO_8859_5, PG_WIN866);
- local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866);
+ converted = local2local(src, dest, len, PG_ISO_8859_5, PG_WIN866, iso2win866, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -572,10 +613,12 @@ win866_to_iso(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN866, PG_ISO_8859_5);
- local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso);
+ converted = local2local(src, dest, len, PG_WIN866, PG_ISO_8859_5, win8662iso, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
index 4d7fb116cfd..88529c644cf 100644
--- a/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/euc2004_sjis2004/euc2004_sjis2004.c
@@ -19,8 +19,8 @@ PG_MODULE_MAGIC;
PG_FUNCTION_INFO_V1(euc_jis_2004_to_shift_jis_2004);
PG_FUNCTION_INFO_V1(shift_jis_2004_to_euc_jis_2004);
-static void euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len);
-static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len);
+static int euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError);
/* ----------
* conv_proc(
@@ -28,8 +28,11 @@ static void shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -39,12 +42,14 @@ euc_jis_2004_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_SHIFT_JIS_2004);
- euc_jis_20042shift_jis_2004(src, dest, len);
+ converted = euc_jis_20042shift_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -53,20 +58,23 @@ shift_jis_2004_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_EUC_JIS_2004);
- shift_jis_20042euc_jis_2004(src, dest, len);
+ converted = shift_jis_20042euc_jis_2004(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_JIS_2004 -> SHIFT_JIS_2004
*/
-static void
-euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
ku,
ten;
@@ -79,8 +87,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -90,8 +102,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
l = pg_encoding_verifymbchar(PG_EUC_JIS_2004, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (c1 == SS2 && l == 2) /* JIS X 0201 kana? */
{
@@ -121,8 +137,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
*p++ = (ku + 0x19b) >> 1;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
if (ku % 2)
@@ -132,8 +152,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
@@ -149,8 +173,12 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ku >= 63 && ku <= 94)
*p++ = (ku + 0x181) >> 1;
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
if (ku % 2)
{
@@ -159,20 +187,30 @@ euc_jis_20042shift_jis_2004(const unsigned char *euc, unsigned char *p, int len)
else if (ten >= 64 && ten <= 94)
*p++ = ten + 0x40;
else
- report_invalid_encoding(PG_EUC_JIS_2004,
+ {
+ if (noError)
+ break;
+ report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
}
else
*p++ = ten + 0x9e;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JIS_2004,
(const char *) euc, len);
+ }
euc += l;
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
@@ -212,9 +250,10 @@ get_ten(int b, int *ku)
* SHIFT_JIS_2004 ---> EUC_JIS_2004
*/
-static void
-shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len)
+static int
+shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1;
int ku,
ten,
@@ -230,8 +269,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -241,8 +284,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
l = pg_encoding_verifymbchar(PG_SHIFT_JIS_2004, (const char *) sjis, len);
if (l < 0 || l > len)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf && l == 1)
{
@@ -266,8 +313,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x100;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xe0 && c1 <= 0xef) /* plane 1 62ku-94ku */
@@ -275,9 +326,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
ku = (c1 << 1) - 0x180;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
-
(const char *) sjis, len);
+ }
ku -= kubun;
}
else if (c1 >= 0xf0 && c1 <= 0xf3) /* plane 2
@@ -286,8 +340,12 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
switch (c1)
{
case 0xf0:
@@ -309,16 +367,24 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
plane = 2;
ten = get_ten(c2, &kubun);
if (ten < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (c1 == 0xf4 && kubun == 1)
ku = 15;
else
ku = (c1 << 1) - 0x19a - kubun;
}
else
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SHIFT_JIS_2004,
(const char *) sjis, len);
+ }
if (plane == 2)
*p++ = SS3;
@@ -330,4 +396,6 @@ shift_jis_20042euc_jis_2004(const unsigned char *sjis, unsigned char *p, int len
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
index e9bb896935f..a8da733803d 100644
--- a/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_cn_and_mic/euc_cn_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_cn2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_cn(const unsigned char *mic, unsigned char *p, int len);
+static int euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_cn_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_cn_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_MULE_INTERNAL);
- euc_cn2mic(src, dest, len);
+ converted = euc_cn2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_CN);
- mic2euc_cn(src, dest, len);
+ converted = mic2euc_cn(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_CN ---> MIC
*/
-static void
-euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_cn2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
while (len > 0)
@@ -76,7 +84,11 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (len < 2 || !IS_HIGHBIT_SET(euc[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = LC_GB2312_80;
*p++ = c1;
*p++ = euc[1];
@@ -86,21 +98,28 @@ euc_cn2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_CN, (const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_CN
*/
-static void
-mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_cn(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
while (len > 0)
@@ -109,11 +128,19 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
if (IS_HIGHBIT_SET(c1))
{
if (c1 != LC_GB2312_80)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_CN,
(const char *) mic, len);
+ }
if (len < 3 || !IS_HIGHBIT_SET(mic[1]) || !IS_HIGHBIT_SET(mic[2]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
mic++;
*p++ = *mic++;
*p++ = *mic++;
@@ -122,12 +149,18 @@ mic2euc_cn(const unsigned char *mic, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
}
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
index 5059f917a98..cc4817dc4cd 100644
--- a/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.c
@@ -42,17 +42,20 @@ PG_FUNCTION_INFO_V1(mic_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void sjis2mic(const unsigned char *sjis, unsigned char *p, int len);
-static void mic2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_jp(const unsigned char *mic, unsigned char *p, int len);
-static void euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len);
-static void sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len);
+static int sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError);
+static int mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_jp2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int sjis2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_jp_to_sjis(PG_FUNCTION_ARGS)
@@ -60,12 +63,14 @@ euc_jp_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_SJIS);
- euc_jp2sjis(src, dest, len);
+ converted = euc_jp2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -74,12 +79,14 @@ sjis_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_EUC_JP);
- sjis2euc_jp(src, dest, len);
+ converted = sjis2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -88,12 +95,14 @@ euc_jp_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_MULE_INTERNAL);
- euc_jp2mic(src, dest, len);
+ converted = euc_jp2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -102,12 +111,14 @@ mic_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_JP);
- mic2euc_jp(src, dest, len);
+ converted = mic2euc_jp(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -116,12 +127,14 @@ sjis_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_MULE_INTERNAL);
- sjis2mic(src, dest, len);
+ converted = sjis2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -130,20 +143,23 @@ mic_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_SJIS);
- mic2sjis(src, dest, len);
+ converted = mic2sjis(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* SJIS ---> MIC
*/
-static void
-sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2mic(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -167,7 +183,11 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
* JIS X0208, X0212, user defined extended characters
*/
if (len < 2 || !ISSJISHEAD(c1) || !ISSJISTAIL(sjis[1]))
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
c2 = sjis[1];
k = (c1 << 8) + c2;
if (k >= 0xed40 && k < 0xf040)
@@ -257,21 +277,28 @@ sjis2mic(const unsigned char *sjis, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS, (const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
}
}
*p = '\0';
+
+ return sjis - start;
}
/*
* MIC ---> SJIS
*/
-static void
-mic2sjis(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2sjis(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1,
c2,
k,
@@ -284,8 +311,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -293,8 +324,12 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
*p++ = mic[1];
else if (c1 == LC_JISX0208)
@@ -350,20 +385,27 @@ mic2sjis(const unsigned char *mic, unsigned char *p, int len)
}
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_SJIS,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP ---> MIC
*/
-static void
-euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -374,8 +416,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -383,8 +429,12 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{ /* 1 byte kana? */
*p++ = LC_JISX0201K;
@@ -406,14 +456,17 @@ euc_jp2mic(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_JP
*/
-static void
-mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_jp(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -424,8 +477,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -433,8 +490,12 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_JISX0201K)
{
*p++ = SS2;
@@ -452,20 +513,27 @@ mic2euc_jp(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_JP,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* EUC_JP -> SJIS
*/
-static void
-euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1,
c2,
k;
@@ -478,8 +546,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
@@ -487,8 +559,12 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_EUC_JP, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_JP,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
/* hankaku kana? */
@@ -551,14 +627,17 @@ euc_jp2sjis(const unsigned char *euc, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return euc - start;
}
/*
* SJIS ---> EUC_JP
*/
-static void
-sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
+static int
+sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = sjis;
int c1,
c2,
i,
@@ -573,8 +652,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
*p++ = c1;
sjis++;
len--;
@@ -582,8 +665,12 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_SJIS, (const char *) sjis, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_SJIS,
(const char *) sjis, len);
+ }
if (c1 >= 0xa1 && c1 <= 0xdf)
{
/* JIS X0201 (1 byte kana) */
@@ -680,4 +767,6 @@ sjis2euc_jp(const unsigned char *sjis, unsigned char *p, int len)
len -= l;
}
*p = '\0';
+
+ return sjis - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
index ac823d6c270..a5b51617e52 100644
--- a/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/euc_kr_and_mic/euc_kr_and_mic.c
@@ -26,13 +26,16 @@ PG_FUNCTION_INFO_V1(mic_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_kr2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_kr(const unsigned char *mic, unsigned char *p, int len);
+static int euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_kr_to_mic(PG_FUNCTION_ARGS)
@@ -40,12 +43,14 @@ euc_kr_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_MULE_INTERNAL);
- euc_kr2mic(src, dest, len);
+ converted = euc_kr2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -54,20 +59,23 @@ mic_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_KR);
- mic2euc_kr(src, dest, len);
+ converted = mic2euc_kr(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_KR ---> MIC
*/
-static void
-euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_kr2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -78,8 +86,12 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_KR, (const char *) euc, len);
if (l != 2)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = LC_KS5601;
*p++ = c1;
*p++ = euc[1];
@@ -89,22 +101,29 @@ euc_kr2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_KR,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_KR
*/
-static void
-mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_kr(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -115,8 +134,12 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -124,18 +147,28 @@ mic2euc_kr(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_KS5601)
{
*p++ = mic[1];
*p++ = mic[2];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_KR,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
index 66c242d7f36..ebb4e0acd53 100644
--- a/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/euc_tw_and_big5/euc_tw_and_big5.c
@@ -32,17 +32,20 @@ PG_FUNCTION_INFO_V1(mic_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
-static void euc_tw2big5(const unsigned char *euc, unsigned char *p, int len);
-static void big52euc_tw(const unsigned char *euc, unsigned char *p, int len);
-static void big52mic(const unsigned char *big5, unsigned char *p, int len);
-static void mic2big5(const unsigned char *mic, unsigned char *p, int len);
-static void euc_tw2mic(const unsigned char *euc, unsigned char *p, int len);
-static void mic2euc_tw(const unsigned char *mic, unsigned char *p, int len);
+static int euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52euc_tw(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError);
+static int mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError);
+static int euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError);
+static int mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError);
Datum
euc_tw_to_big5(PG_FUNCTION_ARGS)
@@ -50,12 +53,14 @@ euc_tw_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_BIG5);
- euc_tw2big5(src, dest, len);
+ converted = euc_tw2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -64,12 +69,14 @@ big5_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_EUC_TW);
- big52euc_tw(src, dest, len);
+ converted = big52euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -78,12 +85,14 @@ euc_tw_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_MULE_INTERNAL);
- euc_tw2mic(src, dest, len);
+ converted = euc_tw2mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -92,12 +101,14 @@ mic_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_EUC_TW);
- mic2euc_tw(src, dest, len);
+ converted = mic2euc_tw(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -106,12 +117,14 @@ big5_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_MULE_INTERNAL);
- big52mic(src, dest, len);
+ converted = big52mic(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -120,21 +133,24 @@ mic_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_BIG5);
- mic2big5(src, dest, len);
+ converted = mic2big5(src, dest, len, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
/*
* EUC_TW ---> Big5
*/
-static void
-euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2big5(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
unsigned char c1;
unsigned short big5buf,
cnsBuf;
@@ -149,8 +165,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Verify and decode the next EUC_TW input character */
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -171,8 +191,12 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
/* Write it out in Big5 */
big5buf = CNStoBIG5(cnsBuf, lc);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_EUC_TW, PG_BIG5,
(const char *) euc, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
@@ -182,22 +206,29 @@ euc_tw2big5(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* Big5 ---> EUC_TW
*/
-static void
-big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52euc_tw(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -212,8 +243,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
@@ -237,8 +272,12 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_EUC_TW,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
@@ -256,14 +295,17 @@ big52euc_tw(const unsigned char *big5, unsigned char *p, int len)
}
}
*p = '\0';
+
+ return big5 - start;
}
/*
* EUC_TW ---> MIC
*/
-static void
-euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
+static int
+euc_tw2mic(const unsigned char *euc, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = euc;
int c1;
int l;
@@ -274,8 +316,12 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
{
l = pg_encoding_verifymbchar(PG_EUC_TW, (const char *) euc, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
if (c1 == SS2)
{
c1 = euc[1]; /* plane No. */
@@ -304,22 +350,29 @@ euc_tw2mic(const unsigned char *euc, unsigned char *p, int len)
else
{ /* should be ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_EUC_TW,
(const char *) euc, len);
+ }
*p++ = c1;
euc++;
len--;
}
}
*p = '\0';
+
+ return euc - start;
}
/*
* MIC ---> EUC_TW
*/
-static void
-mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2euc_tw(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
int c1;
int l;
@@ -330,8 +383,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -339,8 +396,12 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1)
{
*p++ = mic[1];
@@ -362,20 +423,27 @@ mic2euc_tw(const unsigned char *mic, unsigned char *p, int len)
*p++ = mic[3];
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_EUC_TW,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
/*
* Big5 ---> MIC
*/
-static void
-big52mic(const unsigned char *big5, unsigned char *p, int len)
+static int
+big52mic(const unsigned char *big5, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = big5;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -389,8 +457,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
*p++ = c1;
big5++;
len--;
@@ -398,8 +470,12 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_BIG5, (const char *) big5, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_BIG5,
(const char *) big5, len);
+ }
big5buf = (c1 << 8) | big5[1];
cnsBuf = BIG5toCNS(big5buf, &lc);
if (lc != 0)
@@ -412,20 +488,27 @@ big52mic(const unsigned char *big5, unsigned char *p, int len)
*p++ = cnsBuf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_BIG5, PG_MULE_INTERNAL,
(const char *) big5, len);
+ }
big5 += l;
len -= l;
}
*p = '\0';
+
+ return big5 - start;
}
/*
* MIC ---> Big5
*/
-static void
-mic2big5(const unsigned char *mic, unsigned char *p, int len)
+static int
+mic2big5(const unsigned char *mic, unsigned char *p, int len, bool noError)
{
+ const unsigned char *start = mic;
unsigned short c1;
unsigned short big5buf,
cnsBuf;
@@ -438,8 +521,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
{
/* ASCII */
if (c1 == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
*p++ = c1;
mic++;
len--;
@@ -447,8 +534,12 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
l = pg_encoding_verifymbchar(PG_MULE_INTERNAL, (const char *) mic, len);
if (l < 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_MULE_INTERNAL,
(const char *) mic, len);
+ }
if (c1 == LC_CNS11643_1 || c1 == LC_CNS11643_2 || c1 == LCPRV2_B)
{
if (c1 == LCPRV2_B)
@@ -462,16 +553,26 @@ mic2big5(const unsigned char *mic, unsigned char *p, int len)
}
big5buf = CNStoBIG5(cnsBuf, c1);
if (big5buf == 0)
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
*p++ = (big5buf >> 8) & 0x00ff;
*p++ = big5buf & 0x00ff;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_MULE_INTERNAL, PG_BIG5,
(const char *) mic, len);
+ }
mic += l;
len -= l;
}
*p = '\0';
+
+ return mic - start;
}
diff --git a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
index 2e28e6780a5..8610fcb69aa 100644
--- a/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
+++ b/src/backend/utils/mb/conversion_procs/latin2_and_win1250/latin2_and_win1250.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(win1250_to_latin2);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -82,12 +85,14 @@ latin2_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = latin2mic(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -96,12 +101,14 @@ mic_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN2);
- mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2);
+ converted = mic2latin(src, dest, len, LC_ISO8859_2, PG_LATIN2, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -110,13 +117,15 @@ win1250_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_MULE_INTERNAL);
- latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- win1250_2_iso88592);
+ converted = latin2mic_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -125,13 +134,15 @@ mic_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_WIN1250);
- mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
- iso88592_2_win1250);
+ converted = mic2latin_with_table(src, dest, len, LC_ISO8859_2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -140,12 +151,15 @@ latin2_to_win1250(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN2, PG_WIN1250);
- local2local(src, dest, len, PG_LATIN2, PG_WIN1250, iso88592_2_win1250);
+ converted = local2local(src, dest, len, PG_LATIN2, PG_WIN1250,
+ iso88592_2_win1250, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -154,10 +168,13 @@ win1250_to_latin2(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_WIN1250, PG_LATIN2);
- local2local(src, dest, len, PG_WIN1250, PG_LATIN2, win1250_2_iso88592);
+ converted = local2local(src, dest, len, PG_WIN1250, PG_LATIN2,
+ win1250_2_iso88592, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
index bc651410f21..bff27d1c295 100644
--- a/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
+++ b/src/backend/utils/mb/conversion_procs/latin_and_mic/latin_and_mic.c
@@ -30,8 +30,11 @@ PG_FUNCTION_INFO_V1(mic_to_latin4);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -42,12 +45,14 @@ latin1_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = latin2mic(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,12 +61,14 @@ mic_to_latin1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN1);
- mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1);
+ converted = mic2latin(src, dest, len, LC_ISO8859_1, PG_LATIN1, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -70,12 +77,14 @@ latin3_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN3, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = latin2mic(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -84,12 +93,14 @@ mic_to_latin3(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN3);
- mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3);
+ converted = mic2latin(src, dest, len, LC_ISO8859_3, PG_LATIN3, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,12 +109,14 @@ latin4_to_mic(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN4, PG_MULE_INTERNAL);
- latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = latin2mic(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -112,10 +125,12 @@ mic_to_latin4(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_MULE_INTERNAL, PG_LATIN4);
- mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4);
+ converted = mic2latin(src, dest, len, LC_ISO8859_4, PG_LATIN4, noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
index d6067cdc24e..3838b15cab9 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_big5/utf8_and_big5.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_big5);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ big5_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_BIG5, PG_UTF8);
- LocalToUtf(src, len, dest,
- &big5_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = LocalToUtf(src, len, dest,
+ &big5_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_big5(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_BIG5);
- UtfToLocal(src, len, dest,
- &big5_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_BIG5);
+ converted = UtfToLocal(src, len, dest,
+ &big5_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_BIG5,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
index ed90e8e682e..75719fe5f1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_cyrillic/utf8_and_cyrillic.c
@@ -33,8 +33,11 @@ PG_FUNCTION_INFO_V1(koi8u_to_utf8);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -44,16 +47,19 @@ utf8_to_koi8r(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8R);
- UtfToLocal(src, len, dest,
- &koi8r_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = UtfToLocal(src, len, dest,
+ &koi8r_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -62,16 +68,19 @@ koi8r_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8R, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8r_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8R);
+ converted = LocalToUtf(src, len, dest,
+ &koi8r_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8R,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -80,16 +89,19 @@ utf8_to_koi8u(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_KOI8U);
- UtfToLocal(src, len, dest,
- &koi8u_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = UtfToLocal(src, len, dest,
+ &koi8u_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -98,14 +110,17 @@ koi8u_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_KOI8U, PG_UTF8);
- LocalToUtf(src, len, dest,
- &koi8u_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_KOI8U);
+ converted = LocalToUtf(src, len, dest,
+ &koi8u_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_KOI8U,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
index d699affce47..5391001951a 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc2004/utf8_and_euc2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jis_2004_to_unicode_tree,
- LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jis_2004_to_unicode_tree,
+ LUmapEUC_JIS_2004_combined, lengthof(LUmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JIS_2004);
- UtfToLocal(src, len, dest,
- &euc_jis_2004_from_unicode_tree,
- ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
- NULL,
- PG_EUC_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jis_2004_from_unicode_tree,
+ ULmapEUC_JIS_2004_combined, lengthof(ULmapEUC_JIS_2004_combined),
+ NULL,
+ PG_EUC_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
index d7c0ba6a58b..c87d1bf2398 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_cn/utf8_and_euc_cn.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_cn);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_cn_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_CN, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_cn_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = LocalToUtf(src, len, dest,
+ &euc_cn_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_cn(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_CN);
- UtfToLocal(src, len, dest,
- &euc_cn_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_CN);
+ converted = UtfToLocal(src, len, dest,
+ &euc_cn_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_CN,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
index 13a3a23e77b..6a55134db21 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_jp/utf8_and_euc_jp.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_jp);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_jp_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_JP, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_jp_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = LocalToUtf(src, len, dest,
+ &euc_jp_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_jp(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_JP);
- UtfToLocal(src, len, dest,
- &euc_jp_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_JP);
+ converted = UtfToLocal(src, len, dest,
+ &euc_jp_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_JP,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
index 1bbb8aaef7b..fe1924e2fec 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_kr/utf8_and_euc_kr.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_kr);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_kr_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_KR, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_kr_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = LocalToUtf(src, len, dest,
+ &euc_kr_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_kr(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_KR);
- UtfToLocal(src, len, dest,
- &euc_kr_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_KR);
+ converted = UtfToLocal(src, len, dest,
+ &euc_kr_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_KR,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
index 9830045dccd..68215659b57 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_euc_tw/utf8_and_euc_tw.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_euc_tw);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ euc_tw_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_EUC_TW, PG_UTF8);
- LocalToUtf(src, len, dest,
- &euc_tw_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = LocalToUtf(src, len, dest,
+ &euc_tw_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_euc_tw(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_EUC_TW);
- UtfToLocal(src, len, dest,
- &euc_tw_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_EUC_TW);
+ converted = UtfToLocal(src, len, dest,
+ &euc_tw_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_EUC_TW,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
index f86ecf27424..e1a59c39a4d 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gb18030/utf8_and_gb18030.c
@@ -183,8 +183,11 @@ conv_utf8_to_18030(uint32 code)
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -193,16 +196,19 @@ gb18030_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GB18030, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gb18030_to_unicode_tree,
- NULL, 0,
- conv_18030_to_utf8,
- PG_GB18030);
+ converted = LocalToUtf(src, len, dest,
+ &gb18030_to_unicode_tree,
+ NULL, 0,
+ conv_18030_to_utf8,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -211,14 +217,17 @@ utf8_to_gb18030(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GB18030);
- UtfToLocal(src, len, dest,
- &gb18030_from_unicode_tree,
- NULL, 0,
- conv_utf8_to_18030,
- PG_GB18030);
+ converted = UtfToLocal(src, len, dest,
+ &gb18030_from_unicode_tree,
+ NULL, 0,
+ conv_utf8_to_18030,
+ PG_GB18030,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
index 2ab8b16c8a8..881386d5347 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_gbk/utf8_and_gbk.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_gbk);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ gbk_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_GBK, PG_UTF8);
- LocalToUtf(src, len, dest,
- &gbk_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = LocalToUtf(src, len, dest,
+ &gbk_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_gbk(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_GBK);
- UtfToLocal(src, len, dest,
- &gbk_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_GBK);
+ converted = UtfToLocal(src, len, dest,
+ &gbk_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_GBK,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
index 3e49f67ea2f..d93a521badf 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859/utf8_and_iso8859.c
@@ -52,8 +52,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -100,6 +103,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -108,12 +112,15 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -122,7 +129,7 @@ iso8859_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -132,6 +139,7 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -140,12 +148,15 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -154,5 +165,5 @@ utf8_to_iso8859(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for ISO 8859 character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
index 67e713cca11..8ac93604a1b 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_iso8859_1/utf8_and_iso8859_1.c
@@ -26,8 +26,11 @@ PG_FUNCTION_INFO_V1(utf8_to_iso8859_1);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -37,6 +40,8 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c;
CHECK_ENCODING_CONVERSION_ARGS(PG_LATIN1, PG_UTF8);
@@ -45,7 +50,11 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_LATIN1, (const char *) src, len);
+ }
if (!IS_HIGHBIT_SET(c))
*dest++ = c;
else
@@ -58,7 +67,7 @@ iso8859_1_to_utf8(PG_FUNCTION_ARGS)
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
Datum
@@ -67,6 +76,8 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ unsigned char *start = src;
unsigned short c,
c1;
@@ -76,7 +87,11 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
{
c = *src;
if (c == 0)
+ {
+ if (noError)
+ break;
report_invalid_encoding(PG_UTF8, (const char *) src, len);
+ }
/* fast path for ASCII-subset characters */
if (!IS_HIGHBIT_SET(c))
{
@@ -102,11 +117,15 @@ utf8_to_iso8859_1(PG_FUNCTION_ARGS)
len -= 2;
}
else
+ {
+ if (noError)
+ break;
report_untranslatable_char(PG_UTF8, PG_LATIN1,
(const char *) src, len);
+ }
}
}
*dest = '\0';
- PG_RETURN_VOID();
+ PG_RETURN_INT32(src - start);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
index 578f5df4e7f..317daa2d5ee 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_johab/utf8_and_johab.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_johab);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ johab_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_JOHAB, PG_UTF8);
- LocalToUtf(src, len, dest,
- &johab_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = LocalToUtf(src, len, dest,
+ &johab_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_johab(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_JOHAB);
- UtfToLocal(src, len, dest,
- &johab_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_JOHAB);
+ converted = UtfToLocal(src, len, dest,
+ &johab_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_JOHAB,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
index dd9fc2975ad..4c9348aba59 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis/utf8_and_sjis.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_sjis);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ sjis_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SJIS, PG_UTF8);
- LocalToUtf(src, len, dest,
- &sjis_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = LocalToUtf(src, len, dest,
+ &sjis_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_sjis(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SJIS);
- UtfToLocal(src, len, dest,
- &sjis_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_SJIS);
+ converted = UtfToLocal(src, len, dest,
+ &sjis_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_SJIS,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
index 4bcc886d674..1fffdc5930c 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_sjis2004/utf8_and_sjis2004.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_shift_jis_2004);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ shift_jis_2004_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_SHIFT_JIS_2004, PG_UTF8);
- LocalToUtf(src, len, dest,
- &shift_jis_2004_to_unicode_tree,
- LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = LocalToUtf(src, len, dest,
+ &shift_jis_2004_to_unicode_tree,
+ LUmapSHIFT_JIS_2004_combined, lengthof(LUmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_shift_jis_2004(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_SHIFT_JIS_2004);
- UtfToLocal(src, len, dest,
- &shift_jis_2004_from_unicode_tree,
- ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
- NULL,
- PG_SHIFT_JIS_2004);
+ converted = UtfToLocal(src, len, dest,
+ &shift_jis_2004_from_unicode_tree,
+ ULmapSHIFT_JIS_2004_combined, lengthof(ULmapSHIFT_JIS_2004_combined),
+ NULL,
+ PG_SHIFT_JIS_2004,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
index c8e512994a1..d9471dad097 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_uhc/utf8_and_uhc.c
@@ -28,8 +28,11 @@ PG_FUNCTION_INFO_V1(utf8_to_uhc);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
Datum
@@ -38,16 +41,19 @@ uhc_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UHC, PG_UTF8);
- LocalToUtf(src, len, dest,
- &uhc_to_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = LocalToUtf(src, len, dest,
+ &uhc_to_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
Datum
@@ -56,14 +62,17 @@ utf8_to_uhc(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
+ int converted;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, PG_UHC);
- UtfToLocal(src, len, dest,
- &uhc_from_unicode_tree,
- NULL, 0,
- NULL,
- PG_UHC);
+ converted = UtfToLocal(src, len, dest,
+ &uhc_from_unicode_tree,
+ NULL, 0,
+ NULL,
+ PG_UHC,
+ noError);
- PG_RETURN_VOID();
+ PG_RETURN_INT32(converted);
}
diff --git a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
index 0c9493dee56..110ba5677d0 100644
--- a/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
+++ b/src/backend/utils/mb/conversion_procs/utf8_and_win/utf8_and_win.c
@@ -48,8 +48,11 @@ PG_FUNCTION_INFO_V1(utf8_to_win);
* INTEGER, -- destination encoding id
* CSTRING, -- source string (null terminated C string)
* CSTRING, -- destination string (null terminated C string)
- * INTEGER -- source string length
- * ) returns VOID;
+ * INTEGER, -- source string length
+ * BOOL -- if true, don't throw an error if conversion fails
+ * ) returns INTEGER;
+ *
+ * Returns the number of bytes successfully converted.
* ----------
*/
@@ -81,6 +84,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(-1, PG_UTF8);
@@ -89,12 +93,15 @@ win_to_utf8(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- LocalToUtf(src, len, dest,
- maps[i].map1,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = LocalToUtf(src, len, dest,
+ maps[i].map1,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -103,7 +110,7 @@ win_to_utf8(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
Datum
@@ -113,6 +120,7 @@ utf8_to_win(PG_FUNCTION_ARGS)
unsigned char *src = (unsigned char *) PG_GETARG_CSTRING(2);
unsigned char *dest = (unsigned char *) PG_GETARG_CSTRING(3);
int len = PG_GETARG_INT32(4);
+ bool noError = PG_GETARG_BOOL(5);
int i;
CHECK_ENCODING_CONVERSION_ARGS(PG_UTF8, -1);
@@ -121,12 +129,15 @@ utf8_to_win(PG_FUNCTION_ARGS)
{
if (encoding == maps[i].encoding)
{
- UtfToLocal(src, len, dest,
- maps[i].map2,
- NULL, 0,
- NULL,
- encoding);
- PG_RETURN_VOID();
+ int converted;
+
+ converted = UtfToLocal(src, len, dest,
+ maps[i].map2,
+ NULL, 0,
+ NULL,
+ encoding,
+ noError);
+ PG_RETURN_INT32(converted);
}
}
@@ -135,5 +146,5 @@ utf8_to_win(PG_FUNCTION_ARGS)
errmsg("unexpected encoding ID %d for WIN character sets",
encoding)));
- PG_RETURN_VOID();
+ PG_RETURN_INT32(0);
}
diff --git a/src/backend/utils/mb/mbutils.c b/src/backend/utils/mb/mbutils.c
index 2578573b0ab..65753860e35 100644
--- a/src/backend/utils/mb/mbutils.c
+++ b/src/backend/utils/mb/mbutils.c
@@ -406,12 +406,13 @@ pg_do_encoding_conversion(unsigned char *src, int len,
MemoryContextAllocHuge(CurrentMemoryContext,
(Size) len * MAX_CONVERSION_GROWTH + 1);
- OidFunctionCall5(proc,
- Int32GetDatum(src_encoding),
- Int32GetDatum(dest_encoding),
- CStringGetDatum(src),
- CStringGetDatum(result),
- Int32GetDatum(len));
+ (void) OidFunctionCall6(proc,
+ Int32GetDatum(src_encoding),
+ Int32GetDatum(dest_encoding),
+ CStringGetDatum(src),
+ CStringGetDatum(result),
+ Int32GetDatum(len),
+ BoolGetDatum(false));
/*
* If the result is large, it's worth repalloc'ing to release any extra
@@ -849,12 +850,13 @@ pg_unicode_to_server(pg_wchar c, unsigned char *s)
c_as_utf8[c_as_utf8_len] = '\0';
/* Convert, or throw error if we can't */
- FunctionCall5(Utf8ToServerConvProc,
+ FunctionCall6(Utf8ToServerConvProc,
Int32GetDatum(PG_UTF8),
Int32GetDatum(server_encoding),
CStringGetDatum(c_as_utf8),
CStringGetDatum(s),
- Int32GetDatum(c_as_utf8_len));
+ Int32GetDatum(c_as_utf8_len),
+ BoolGetDatum(false));
}
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 43fc297eb69..ee6be95b08d 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -28,6 +28,7 @@ static void check_for_reg_data_type_usage(ClusterInfo *cluster);
static void check_for_jsonb_9_4_usage(ClusterInfo *cluster);
static void check_for_pg_role_prefix(ClusterInfo *cluster);
static void check_for_new_tablespace_dir(ClusterInfo *new_cluster);
+static void check_for_user_defined_encoding_conversions(ClusterInfo *cluster);
static char *get_canonical_locale_name(int category, const char *locale);
@@ -102,6 +103,15 @@ check_and_dump_old_cluster(bool live_check)
check_for_reg_data_type_usage(&old_cluster);
check_for_isn_and_int8_passing_mismatch(&old_cluster);
+ /*
+ * PG 14 changed the function signature of encoding conversion functions.
+ * Conversions from older versions cannot be upgraded automatically
+ * because the user-defined functions used by the encoding conversions
+ * need to changed to match the new signature.
+ */
+ if (GET_MAJOR_VERSION(old_cluster.major_version) <= 1300)
+ check_for_user_defined_encoding_conversions(&old_cluster);
+
/*
* Pre-PG 14 allowed user defined postfix operators, which are not
* supported anymore. Verify there are none, iff applicable.
@@ -1268,6 +1278,91 @@ check_for_pg_role_prefix(ClusterInfo *cluster)
check_ok();
}
+/*
+ * Verify that no user-defined encoding conversions exist.
+ */
+static void
+check_for_user_defined_encoding_conversions(ClusterInfo *cluster)
+{
+ int dbnum;
+ FILE *script = NULL;
+ bool found = false;
+ char output_path[MAXPGPATH];
+
+ prep_status("Checking for user-defined encoding conversions");
+
+ snprintf(output_path, sizeof(output_path),
+ "encoding_conversions.txt");
+
+ /* Find any user defined encoding conversions */
+ for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
+ {
+ PGresult *res;
+ bool db_used = false;
+ int ntups;
+ int rowno;
+ int i_conoid,
+ i_conname,
+ i_nspname;
+ DbInfo *active_db = &cluster->dbarr.dbs[dbnum];
+ PGconn *conn = connectToServer(cluster, active_db->db_name);
+
+ /*
+ * The query below hardcodes FirstNormalObjectId as 16384 rather than
+ * interpolating that C #define into the query because, if that
+ * #define is ever changed, the cutoff we want to use is the value
+ * used by pre-version 14 servers, not that of some future version.
+ */
+ res = executeQueryOrDie(conn,
+ "SELECT c.oid as conoid, c.conname, n.nspname "
+ "FROM pg_catalog.pg_conversion c, "
+ " pg_catalog.pg_namespace n "
+ "WHERE c.connamespace = n.oid AND "
+ " c.oid >= 16384");
+ ntups = PQntuples(res);
+ i_conoid = PQfnumber(res, "conoid");
+ i_conname = PQfnumber(res, "conname");
+ i_nspname = PQfnumber(res, "nspname");
+ for (rowno = 0; rowno < ntups; rowno++)
+ {
+ found = true;
+ if (script == NULL &&
+ (script = fopen_priv(output_path, "w")) == NULL)
+ pg_fatal("could not open file \"%s\": %s\n",
+ output_path, strerror(errno));
+ if (!db_used)
+ {
+ fprintf(script, "In database: %s\n", active_db->db_name);
+ db_used = true;
+ }
+ fprintf(script, " (oid=%s) %s.%s\n",
+ PQgetvalue(res, rowno, i_conoid),
+ PQgetvalue(res, rowno, i_nspname),
+ PQgetvalue(res, rowno, i_conname));
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+ }
+
+ if (script)
+ fclose(script);
+
+ if (found)
+ {
+ pg_log(PG_REPORT, "fatal\n");
+ pg_fatal("Your installation contains user-defined encoding conversions.\n"
+ "The conversion function parameters changed in PostgreSQL version 14\n"
+ "so this cluster cannot currently be upgraded. You can remove the\n"
+ "encoding conversions in the old cluster and restart the upgrade.\n"
+ "A list of user-defined encoding conversions is in the file:\n"
+ " %s\n\n", output_path);
+ }
+ else
+ check_ok();
+}
+
/*
* get_canonical_locale_name
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f8174061ef3..75b829e8514 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10766,388 +10766,388 @@
# conversion functions
{ oid => '4302',
descr => 'internal conversion function for KOI8R to MULE_INTERNAL',
- proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_mic',
+ proname => 'koi8r_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4303',
descr => 'internal conversion function for MULE_INTERNAL to KOI8R',
- proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_koi8r',
+ proname => 'mic_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4304',
descr => 'internal conversion function for ISO-8859-5 to MULE_INTERNAL',
- proname => 'iso_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_mic',
+ proname => 'iso_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4305',
descr => 'internal conversion function for MULE_INTERNAL to ISO-8859-5',
- proname => 'mic_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_iso',
+ proname => 'mic_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4306',
descr => 'internal conversion function for WIN1251 to MULE_INTERNAL',
- proname => 'win1251_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_mic',
+ proname => 'win1251_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4307',
descr => 'internal conversion function for MULE_INTERNAL to WIN1251',
- proname => 'mic_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1251',
+ proname => 'mic_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4308',
descr => 'internal conversion function for WIN866 to MULE_INTERNAL',
- proname => 'win866_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_mic',
+ proname => 'win866_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_mic',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4309',
descr => 'internal conversion function for MULE_INTERNAL to WIN866',
- proname => 'mic_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win866',
+ proname => 'mic_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4310', descr => 'internal conversion function for KOI8R to WIN1251',
- proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'koi8r_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'koi8r_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4311', descr => 'internal conversion function for WIN1251 to KOI8R',
- proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_koi8r', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4312', descr => 'internal conversion function for KOI8R to WIN866',
- proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_win866',
+ proname => 'koi8r_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4313', descr => 'internal conversion function for WIN866 to KOI8R',
- proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_koi8r',
+ proname => 'win866_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4314',
descr => 'internal conversion function for WIN866 to WIN1251',
- proname => 'win866_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win866_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win866_to_win1251', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4315',
descr => 'internal conversion function for WIN1251 to WIN866',
- proname => 'win1251_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1251_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1251_to_win866', probin => '$libdir/cyrillic_and_mic' },
{ oid => '4316',
descr => 'internal conversion function for ISO-8859-5 to KOI8R',
- proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_koi8r',
+ proname => 'iso_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_koi8r',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4317',
descr => 'internal conversion function for KOI8R to ISO-8859-5',
- proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_iso',
+ proname => 'koi8r_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4318',
descr => 'internal conversion function for ISO-8859-5 to WIN1251',
- proname => 'iso_to_win1251', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win1251',
+ proname => 'iso_to_win1251', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win1251',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4319',
descr => 'internal conversion function for WIN1251 to ISO-8859-5',
- proname => 'win1251_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1251_to_iso',
+ proname => 'win1251_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1251_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4320',
descr => 'internal conversion function for ISO-8859-5 to WIN866',
- proname => 'iso_to_win866', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso_to_win866',
+ proname => 'iso_to_win866', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso_to_win866',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4321',
descr => 'internal conversion function for WIN866 to ISO-8859-5',
- proname => 'win866_to_iso', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win866_to_iso',
+ proname => 'win866_to_iso', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win866_to_iso',
probin => '$libdir/cyrillic_and_mic' },
{ oid => '4322',
descr => 'internal conversion function for EUC_CN to MULE_INTERNAL',
- proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_mic',
+ proname => 'euc_cn_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_mic',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4323',
descr => 'internal conversion function for MULE_INTERNAL to EUC_CN',
- proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_cn',
+ proname => 'mic_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_cn',
probin => '$libdir/euc_cn_and_mic' },
{ oid => '4324', descr => 'internal conversion function for EUC_JP to SJIS',
- proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_sjis',
+ proname => 'euc_jp_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4325', descr => 'internal conversion function for SJIS to EUC_JP',
- proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_euc_jp',
+ proname => 'sjis_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4326',
descr => 'internal conversion function for EUC_JP to MULE_INTERNAL',
- proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_mic',
+ proname => 'euc_jp_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4327',
descr => 'internal conversion function for SJIS to MULE_INTERNAL',
- proname => 'sjis_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_mic',
+ proname => 'sjis_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_mic',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4328',
descr => 'internal conversion function for MULE_INTERNAL to EUC_JP',
- proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_jp',
+ proname => 'mic_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_jp',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4329',
descr => 'internal conversion function for MULE_INTERNAL to SJIS',
- proname => 'mic_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_sjis',
+ proname => 'mic_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_sjis',
probin => '$libdir/euc_jp_and_sjis' },
{ oid => '4330',
descr => 'internal conversion function for EUC_KR to MULE_INTERNAL',
- proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_mic',
+ proname => 'euc_kr_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_mic',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4331',
descr => 'internal conversion function for MULE_INTERNAL to EUC_KR',
- proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_kr',
+ proname => 'mic_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_kr',
probin => '$libdir/euc_kr_and_mic' },
{ oid => '4332', descr => 'internal conversion function for EUC_TW to BIG5',
- proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_big5',
+ proname => 'euc_tw_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4333', descr => 'internal conversion function for BIG5 to EUC_TW',
- proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_euc_tw',
+ proname => 'big5_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4334',
descr => 'internal conversion function for EUC_TW to MULE_INTERNAL',
- proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_mic',
+ proname => 'euc_tw_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4335',
descr => 'internal conversion function for BIG5 to MULE_INTERNAL',
- proname => 'big5_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_mic',
+ proname => 'big5_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_mic',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4336',
descr => 'internal conversion function for MULE_INTERNAL to EUC_TW',
- proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_euc_tw',
+ proname => 'mic_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_euc_tw',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4337',
descr => 'internal conversion function for MULE_INTERNAL to BIG5',
- proname => 'mic_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_big5',
+ proname => 'mic_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_big5',
probin => '$libdir/euc_tw_and_big5' },
{ oid => '4338',
descr => 'internal conversion function for LATIN2 to MULE_INTERNAL',
- proname => 'latin2_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin2_to_mic',
+ proname => 'latin2_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin2_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4339',
descr => 'internal conversion function for MULE_INTERNAL to LATIN2',
- proname => 'mic_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin2',
+ proname => 'mic_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin2',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4340',
descr => 'internal conversion function for WIN1250 to MULE_INTERNAL',
- proname => 'win1250_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win1250_to_mic',
+ proname => 'win1250_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win1250_to_mic',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4341',
descr => 'internal conversion function for MULE_INTERNAL to WIN1250',
- proname => 'mic_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_win1250',
+ proname => 'mic_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_win1250',
probin => '$libdir/latin2_and_win1250' },
{ oid => '4342',
descr => 'internal conversion function for LATIN2 to WIN1250',
- proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'latin2_to_win1250', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'latin2_to_win1250', probin => '$libdir/latin2_and_win1250' },
{ oid => '4343',
descr => 'internal conversion function for WIN1250 to LATIN2',
- proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'win1250_to_latin2', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'win1250_to_latin2', probin => '$libdir/latin2_and_win1250' },
{ oid => '4344',
descr => 'internal conversion function for LATIN1 to MULE_INTERNAL',
- proname => 'latin1_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin1_to_mic',
+ proname => 'latin1_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin1_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4345',
descr => 'internal conversion function for MULE_INTERNAL to LATIN1',
- proname => 'mic_to_latin1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin1',
+ proname => 'mic_to_latin1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin1',
probin => '$libdir/latin_and_mic' },
{ oid => '4346',
descr => 'internal conversion function for LATIN3 to MULE_INTERNAL',
- proname => 'latin3_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin3_to_mic',
+ proname => 'latin3_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin3_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4347',
descr => 'internal conversion function for MULE_INTERNAL to LATIN3',
- proname => 'mic_to_latin3', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin3',
+ proname => 'mic_to_latin3', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin3',
probin => '$libdir/latin_and_mic' },
{ oid => '4348',
descr => 'internal conversion function for LATIN4 to MULE_INTERNAL',
- proname => 'latin4_to_mic', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'latin4_to_mic',
+ proname => 'latin4_to_mic', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'latin4_to_mic',
probin => '$libdir/latin_and_mic' },
{ oid => '4349',
descr => 'internal conversion function for MULE_INTERNAL to LATIN4',
- proname => 'mic_to_latin4', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'mic_to_latin4',
+ proname => 'mic_to_latin4', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'mic_to_latin4',
probin => '$libdir/latin_and_mic' },
{ oid => '4352', descr => 'internal conversion function for BIG5 to UTF8',
- proname => 'big5_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'big5_to_utf8',
+ proname => 'big5_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'big5_to_utf8',
probin => '$libdir/utf8_and_big5' },
{ oid => '4353', descr => 'internal conversion function for UTF8 to BIG5',
- proname => 'utf8_to_big5', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_big5',
+ proname => 'utf8_to_big5', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_big5',
probin => '$libdir/utf8_and_big5' },
{ oid => '4354', descr => 'internal conversion function for UTF8 to KOI8R',
- proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8r',
+ proname => 'utf8_to_koi8r', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8r',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4355', descr => 'internal conversion function for KOI8R to UTF8',
- proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8r_to_utf8',
+ proname => 'koi8r_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8r_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4356', descr => 'internal conversion function for UTF8 to KOI8U',
- proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_koi8u',
+ proname => 'utf8_to_koi8u', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_koi8u',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4357', descr => 'internal conversion function for KOI8U to UTF8',
- proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'koi8u_to_utf8',
+ proname => 'koi8u_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'koi8u_to_utf8',
probin => '$libdir/utf8_and_cyrillic' },
{ oid => '4358', descr => 'internal conversion function for UTF8 to WIN',
- proname => 'utf8_to_win', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_win',
+ proname => 'utf8_to_win', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_win',
probin => '$libdir/utf8_and_win' },
{ oid => '4359', descr => 'internal conversion function for WIN to UTF8',
- proname => 'win_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'win_to_utf8',
+ proname => 'win_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'win_to_utf8',
probin => '$libdir/utf8_and_win' },
{ oid => '4360', descr => 'internal conversion function for EUC_CN to UTF8',
- proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_cn_to_utf8',
+ proname => 'euc_cn_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_cn_to_utf8',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4361', descr => 'internal conversion function for UTF8 to EUC_CN',
- proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_cn',
+ proname => 'utf8_to_euc_cn', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_cn',
probin => '$libdir/utf8_and_euc_cn' },
{ oid => '4362', descr => 'internal conversion function for EUC_JP to UTF8',
- proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_jp_to_utf8',
+ proname => 'euc_jp_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_jp_to_utf8',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4363', descr => 'internal conversion function for UTF8 to EUC_JP',
- proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_jp',
+ proname => 'utf8_to_euc_jp', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_jp',
probin => '$libdir/utf8_and_euc_jp' },
{ oid => '4364', descr => 'internal conversion function for EUC_KR to UTF8',
- proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_kr_to_utf8',
+ proname => 'euc_kr_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_kr_to_utf8',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4365', descr => 'internal conversion function for UTF8 to EUC_KR',
- proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_kr',
+ proname => 'utf8_to_euc_kr', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_kr',
probin => '$libdir/utf8_and_euc_kr' },
{ oid => '4366', descr => 'internal conversion function for EUC_TW to UTF8',
- proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'euc_tw_to_utf8',
+ proname => 'euc_tw_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'euc_tw_to_utf8',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4367', descr => 'internal conversion function for UTF8 to EUC_TW',
- proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_euc_tw',
+ proname => 'utf8_to_euc_tw', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_euc_tw',
probin => '$libdir/utf8_and_euc_tw' },
{ oid => '4368', descr => 'internal conversion function for GB18030 to UTF8',
- proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gb18030_to_utf8',
+ proname => 'gb18030_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gb18030_to_utf8',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4369', descr => 'internal conversion function for UTF8 to GB18030',
- proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gb18030',
+ proname => 'utf8_to_gb18030', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gb18030',
probin => '$libdir/utf8_and_gb18030' },
{ oid => '4370', descr => 'internal conversion function for GBK to UTF8',
- proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'gbk_to_utf8',
+ proname => 'gbk_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'gbk_to_utf8',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4371', descr => 'internal conversion function for UTF8 to GBK',
- proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_gbk',
+ proname => 'utf8_to_gbk', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_gbk',
probin => '$libdir/utf8_and_gbk' },
{ oid => '4372',
descr => 'internal conversion function for UTF8 to ISO-8859 2-16',
- proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_iso8859',
+ proname => 'utf8_to_iso8859', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_iso8859',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4373',
descr => 'internal conversion function for ISO-8859 2-16 to UTF8',
- proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'iso8859_to_utf8',
+ proname => 'iso8859_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'iso8859_to_utf8',
probin => '$libdir/utf8_and_iso8859' },
{ oid => '4374', descr => 'internal conversion function for LATIN1 to UTF8',
- proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'iso8859_1_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'iso8859_1_to_utf8', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4375', descr => 'internal conversion function for UTF8 to LATIN1',
- proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_iso8859_1', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_iso8859_1', probin => '$libdir/utf8_and_iso8859_1' },
{ oid => '4376', descr => 'internal conversion function for JOHAB to UTF8',
- proname => 'johab_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'johab_to_utf8',
+ proname => 'johab_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'johab_to_utf8',
probin => '$libdir/utf8_and_johab' },
{ oid => '4377', descr => 'internal conversion function for UTF8 to JOHAB',
- proname => 'utf8_to_johab', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_johab',
+ proname => 'utf8_to_johab', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_johab',
probin => '$libdir/utf8_and_johab' },
{ oid => '4378', descr => 'internal conversion function for SJIS to UTF8',
- proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'sjis_to_utf8',
+ proname => 'sjis_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'sjis_to_utf8',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4379', descr => 'internal conversion function for UTF8 to SJIS',
- proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_sjis',
+ proname => 'utf8_to_sjis', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_sjis',
probin => '$libdir/utf8_and_sjis' },
{ oid => '4380', descr => 'internal conversion function for UHC to UTF8',
- proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'uhc_to_utf8',
+ proname => 'uhc_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'uhc_to_utf8',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4381', descr => 'internal conversion function for UTF8 to UHC',
- proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4', prosrc => 'utf8_to_uhc',
+ proname => 'utf8_to_uhc', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool', prosrc => 'utf8_to_uhc',
probin => '$libdir/utf8_and_uhc' },
{ oid => '4382',
descr => 'internal conversion function for EUC_JIS_2004 to UTF8',
- proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'euc_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_utf8', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4383',
descr => 'internal conversion function for UTF8 to EUC_JIS_2004',
- proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_euc_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_euc_jis_2004', probin => '$libdir/utf8_and_euc2004' },
{ oid => '4384',
descr => 'internal conversion function for SHIFT_JIS_2004 to UTF8',
- proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'shift_jis_2004_to_utf8', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_utf8', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4385',
descr => 'internal conversion function for UTF8 to SHIFT_JIS_2004',
- proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'void',
- proargtypes => 'int4 int4 cstring internal int4',
+ proname => 'utf8_to_shift_jis_2004', prolang => 'c', prorettype => 'int4',
+ proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'utf8_to_shift_jis_2004', probin => '$libdir/utf8_and_sjis2004' },
{ oid => '4386',
descr => 'internal conversion function for EUC_JIS_2004 to SHIFT_JIS_2004',
proname => 'euc_jis_2004_to_shift_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'euc_jis_2004_to_shift_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
{ oid => '4387',
descr => 'internal conversion function for SHIFT_JIS_2004 to EUC_JIS_2004',
proname => 'shift_jis_2004_to_euc_jis_2004', prolang => 'c',
- prorettype => 'void', proargtypes => 'int4 int4 cstring internal int4',
+ prorettype => 'int4', proargtypes => 'int4 int4 cstring internal int4 bool',
prosrc => 'shift_jis_2004_to_euc_jis_2004',
probin => '$libdir/euc2004_sjis2004' },
diff --git a/src/include/mb/pg_wchar.h b/src/include/mb/pg_wchar.h
index 64b22e4b0d4..346a41a1f3d 100644
--- a/src/include/mb/pg_wchar.h
+++ b/src/include/mb/pg_wchar.h
@@ -627,18 +627,18 @@ extern void pg_unicode_to_server(pg_wchar c, unsigned char *s);
extern unsigned short BIG5toCNS(unsigned short big5, unsigned char *lc);
extern unsigned short CNStoBIG5(unsigned short cns, unsigned char lc);
-extern void UtfToLocal(const unsigned char *utf, int len,
- unsigned char *iso,
- const pg_mb_radix_tree *map,
- const pg_utf_to_local_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
-extern void LocalToUtf(const unsigned char *iso, int len,
- unsigned char *utf,
- const pg_mb_radix_tree *map,
- const pg_local_to_utf_combined *cmap, int cmapsize,
- utf_local_conversion_func conv_func,
- int encoding);
+extern int UtfToLocal(const unsigned char *utf, int len,
+ unsigned char *iso,
+ const pg_mb_radix_tree *map,
+ const pg_utf_to_local_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
+extern int LocalToUtf(const unsigned char *iso, int len,
+ unsigned char *utf,
+ const pg_mb_radix_tree *map,
+ const pg_local_to_utf_combined *cmap, int cmapsize,
+ utf_local_conversion_func conv_func,
+ int encoding, bool noError);
extern bool pg_verifymbstr(const char *mbstr, int len, bool noError);
extern bool pg_verify_mbstr(int encoding, const char *mbstr, int len,
@@ -656,18 +656,19 @@ extern void report_invalid_encoding(int encoding, const char *mbstr, int len) pg
extern void report_untranslatable_char(int src_encoding, int dest_encoding,
const char *mbstr, int len) pg_attribute_noreturn();
-extern void local2local(const unsigned char *l, unsigned char *p, int len,
- int src_encoding, int dest_encoding, const unsigned char *tab);
-extern void latin2mic(const unsigned char *l, unsigned char *p, int len,
- int lc, int encoding);
-extern void mic2latin(const unsigned char *mic, unsigned char *p, int len,
- int lc, int encoding);
-extern void latin2mic_with_table(const unsigned char *l, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
-extern void mic2latin_with_table(const unsigned char *mic, unsigned char *p,
- int len, int lc, int encoding,
- const unsigned char *tab);
+extern int local2local(const unsigned char *l, unsigned char *p, int len,
+ int src_encoding, int dest_encoding, const unsigned char *tab,
+ bool noError);
+extern int latin2mic(const unsigned char *l, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int mic2latin(const unsigned char *mic, unsigned char *p, int len,
+ int lc, int encoding, bool noError);
+extern int latin2mic_with_table(const unsigned char *l, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
+extern int mic2latin_with_table(const unsigned char *mic, unsigned char *p,
+ int len, int lc, int encoding,
+ const unsigned char *tab, bool noError);
#ifdef WIN32
extern WCHAR *pgwin32_message_to_UTF16(const char *str, int len, int *utf16len);
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 254ca06d3dd..23ba60e395f 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1052,13 +1052,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
oid | proname | oid | conname
-----+---------+-----+---------
(0 rows)
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index bbd3834b634..04691745981 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -556,13 +556,14 @@ WHERE p1.conproc = 0 OR
SELECT p.oid, p.proname, c.oid, c.conname
FROM pg_proc p, pg_conversion c
WHERE p.oid = c.conproc AND
- (p.prorettype != 'void'::regtype OR p.proretset OR
- p.pronargs != 5 OR
+ (p.prorettype != 'int4'::regtype OR p.proretset OR
+ p.pronargs != 6 OR
p.proargtypes[0] != 'int4'::regtype OR
p.proargtypes[1] != 'int4'::regtype OR
p.proargtypes[2] != 'cstring'::regtype OR
p.proargtypes[3] != 'internal'::regtype OR
- p.proargtypes[4] != 'int4'::regtype);
+ p.proargtypes[4] != 'int4'::regtype OR
+ p.proargtypes[5] != 'bool'::regtype);
-- Check for conprocs that don't perform the specific conversion that
-- pg_conversion alleges they do, by trying to invoke each conversion
--
2.29.2
--------------39E46EF15EBFAA37D53332E5
Content-Type: text/x-patch; charset=UTF-8;
name="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename*0="v3-0003-Add-tests-for-the-new-noError-variants-of-built-i.pa";
filename*1="tch"
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-03-04 17:37 Ibrar Ahmed <[email protected]>
parent: Kyotaro Horiguchi <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Ibrar Ahmed @ 2021-03-04 17:37 UTC (permalink / raw)
To: [email protected]; Kyotaro Horiguchi <[email protected]>; +Cc: Andres Freund <[email protected]>; [email protected]; Masahiko Sawada <[email protected]>; pgsql-hackers; [email protected]
On Wed, Jan 6, 2021 at 1:33 PM Kyotaro Horiguchi <[email protected]>
wrote:
> At Mon, 17 Aug 2020 11:22:15 -0700, Andres Freund <[email protected]>
> wrote in
> > Hi,
> >
> > On 2020-08-17 14:05:37 +0300, Heikki Linnakangas wrote:
> > > On 14/04/2020 22:04, Teja Mupparti wrote:
> > > > Thanks Kyotaro and Masahiko for the feedback. I think there is a
> > > > consensus on the critical-section around truncate,
> > >
> > > +1
> >
> > I'm inclined to think that we should do that independent of the far more
> > complicated fix for other related issues.
> ...
> > > Perhaps a better approach would be to prevent the checkpoint from
> > > completing, until all in-progress truncations have completed. We have a
> > > mechanism to wait out in-progress commits at the beginning of a
> checkpoint,
> > > right after the redo point has been established. See comments around
> the
> > > GetVirtualXIDsDelayingChkpt() function call in CreateCheckPoint(). We
> could
> > > have a similar mechanism to wait out the truncations before
> *completing* a
> > > checkpoint.
> >
> > What I outlined earlier *is* essentially a way to do so, by preventing
> > checkpointing from finishing the buffer scan while a dangerous state
> > exists.
>
> Seems reasonable. The attached does that. It actually works for the
> initial case.
>
> regards.
>
> --
> Kyotaro Horiguchi
> NTT Open Source Software Center
>
The regression is failing for this patch, do you mind look at that and send
the updated patch?
https://api.cirrus-ci.com/v1/task/6313174510075904/logs/test.log
...
t/006_logical_decoding.pl ............ ok
t/007_sync_rep.pl .................... ok
Bailout called. Further testing stopped: system pg_ctl failed
FAILED--Further testing stopped: system pg_ctl failed
make[2]: *** [Makefile:19: check] Error 255
make[1]: *** [Makefile:49: check-recovery-recurse] Error 2
make: *** [GNUmakefile:71: check-world-src/test-recurse] Error 2
...
--
Ibrar Ahmed
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-03-05 03:01 Kyotaro Horiguchi <[email protected]>
parent: Ibrar Ahmed <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Kyotaro Horiguchi @ 2021-03-05 03:01 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]
At Thu, 4 Mar 2021 22:37:23 +0500, Ibrar Ahmed <[email protected]> wrote in
> The regression is failing for this patch, do you mind look at that and send
> the updated patch?
>
> https://api.cirrus-ci.com/v1/task/6313174510075904/logs/test.log
>
> ...
> t/006_logical_decoding.pl ............ ok
> t/007_sync_rep.pl .................... ok
> Bailout called. Further testing stopped: system pg_ctl failed
> FAILED--Further testing stopped: system pg_ctl failed
> make[2]: *** [Makefile:19: check] Error 255
> make[1]: *** [Makefile:49: check-recovery-recurse] Error 2
> make: *** [GNUmakefile:71: check-world-src/test-recurse] Error 2
> ...
(I regret that I sent this as .patch file..)
Thaks for pointing that!
The patch assumed that CHKPT_START/COMPLETE barrier are exclusively
used each other, but MarkBufferDirtyHint which delays checkpoint start
is called in RelationTruncate while delaying checkpoint completion.
That is not a strange nor harmful behavior. I changed delayChkpt to a
bitmap integer from an enum so that both barrier are separately
triggered.
I'm not sure this is the way to go here, though. This fixes the issue
of a crash during RelationTruncate, but the issue of smgrtruncate
failure during RelationTruncate still remains (unless we treat that
failure as PANIC?).
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1fa1..c1b0b48362 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3072,8 +3072,8 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
* crash/basebackup, even though the state of the data directory would
* require it.
*/
- Assert(!MyProc->delayChkpt);
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/* WAL log truncation */
WriteMTruncateXlogRec(newOldestMultiDB,
@@ -3099,7 +3099,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
/* Then offsets */
PerformOffsetsTruncation(oldestMulti, newOldestMulti);
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
LWLockRelease(MultiXactTruncationLock);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20d6c..85c720491b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -463,7 +463,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
proc->lxid = (LocalTransactionId) xid;
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
- proc->delayChkpt = false;
+ proc->delayChkpt = 0;
proc->statusFlags = 0;
proc->pid = 0;
proc->backendId = InvalidBackendId;
@@ -1109,7 +1109,8 @@ EndPrepare(GlobalTransaction gxact)
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
XLogBeginInsert();
for (record = records.head; record != NULL; record = record->next)
@@ -1152,7 +1153,7 @@ EndPrepare(GlobalTransaction gxact)
* checkpoint starting after this will certainly see the gxact as a
* candidate for fsyncing.
*/
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
/*
* Remember that we have this GlobalTransaction entry locked for us. If
@@ -2198,7 +2199,8 @@ RecordTransactionCommitPrepared(TransactionId xid,
START_CRIT_SECTION();
/* See notes in RecordTransactionCommit */
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/*
* Emit the XLOG commit record. Note that we mark 2PC commits as
@@ -2246,7 +2248,7 @@ RecordTransactionCommitPrepared(TransactionId xid,
TransactionIdCommitTree(xid, nchildren, children);
/* Checkpoint can proceed now */
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 4e6a3df6b8..f033e8940a 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1334,8 +1334,9 @@ RecordTransactionCommit(void)
* This makes checkpoint's determination of which xacts are delayChkpt
* a bit fuzzy, but it doesn't matter.
*/
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
SetCurrentTransactionStopTimestamp();
@@ -1436,7 +1437,7 @@ RecordTransactionCommit(void)
*/
if (markXidCommitted)
{
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
}
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 377afb8732..5f5703bd57 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9065,18 +9065,30 @@ CreateCheckPoint(int flags)
* and we will correctly flush the update below. So we cannot miss any
* xacts we need to wait for.
*/
- vxids = GetVirtualXIDsDelayingChkpt(&nvxids);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_START);
if (nvxids > 0)
{
do
{
pg_usleep(10000L); /* wait for 10 msec */
- } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids));
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_START));
}
pfree(vxids);
CheckPointGuts(checkPoint.redo, flags);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_COMPLETE);
+ if (nvxids > 0)
+ {
+ do
+ {
+ pg_usleep(10000L); /* wait for 10 msec */
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_COMPLETE));
+ }
+ pfree(vxids);
+
/*
* Take a snapshot of running transactions and write this to WAL. This
* allows us to reconstruct the state of running transactions during
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 7052dc245e..1edd1b67ff 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -923,7 +923,7 @@ XLogSaveBufferForHint(Buffer buffer, bool buffer_std)
/*
* Ensure no checkpoint can change our view of RedoRecPtr.
*/
- Assert(MyProc->delayChkpt);
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) != 0);
/*
* Update RedoRecPtr so that we can make the right decision
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index cba7a9ada0..579f23c991 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -325,6 +325,16 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
RelationPreTruncate(rel);
+ /*
+ * If the file truncation fails but the concurrent checkpoint completes
+ * just before that, the next crash recovery can fail due to WAL records
+ * inconsistent with the untruncated pages. To avoid that situation we
+ * delay the checkpoint completion until we confirm the truncation to be
+ * successful.
+ */
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_COMPLETE) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_COMPLETE;
+
/*
* We WAL-log the truncation before actually truncating, which means
* trouble if the truncation fails. If we then crash, the WAL replay
@@ -373,6 +383,8 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
*/
if (need_fsm_vacuum)
FreeSpaceMapVacuumRange(rel, nblocks, InvalidBlockNumber);
+
+ MyProc->delayChkpt &= ~DELAY_CHKPT_COMPLETE;
}
/*
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092..1c9e971b31 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3803,7 +3803,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
{
XLogRecPtr lsn = InvalidXLogRecPtr;
bool dirtied = false;
- bool delayChkpt = false;
+ int delayChkptMask = ~0;
uint32 buf_state;
/*
@@ -3853,7 +3853,9 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
* essential that CreateCheckpoint waits for virtual transactions
* rather than full transactionids.
*/
- MyProc->delayChkpt = delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
+ delayChkptMask = ~DELAY_CHKPT_START;
lsn = XLogSaveBufferForHint(buffer, buffer_std);
}
@@ -3885,8 +3887,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
buf_state |= BM_DIRTY | BM_JUST_DIRTIED;
UnlockBufHdr(bufHdr, buf_state);
- if (delayChkpt)
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= delayChkptMask;
if (dirtied)
{
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 4fc6ffb917..3e6759886a 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -655,7 +655,10 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -694,7 +697,10 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
proc->xid = InvalidTransactionId;
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -2955,7 +2961,8 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* delaying checkpoint because they have critical actions in progress.
*
* Constructs an array of VXIDs of transactions that are currently in commit
- * critical sections, as shown by having delayChkpt set in their PGPROC.
+ * critical sections, as shown by having delayChkpt set to the specified value
+ * in their PGPROC.
*
* Returns a palloc'd array that should be freed by the caller.
* *nvxids is the number of valid entries.
@@ -2969,13 +2976,15 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* for clearing of delayChkpt to propagate is unimportant for correctness.
*/
VirtualTransactionId *
-GetVirtualXIDsDelayingChkpt(int *nvxids)
+GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
{
VirtualTransactionId *vxids;
ProcArrayStruct *arrayP = procArray;
int count = 0;
int index;
+ Assert(type != 0);
+
/* allocate what's certainly enough result space */
vxids = (VirtualTransactionId *)
palloc(sizeof(VirtualTransactionId) * arrayP->maxProcs);
@@ -2987,7 +2996,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];
- if (proc->delayChkpt)
+ if ((proc->delayChkpt & type) != 0)
{
VirtualTransactionId vxid;
@@ -3013,12 +3022,14 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
* those numbers should be small enough for it not to be a problem.
*/
bool
-HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
+HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
{
bool result = false;
ProcArrayStruct *arrayP = procArray;
int index;
+ Assert(type != 0);
+
LWLockAcquire(ProcArrayLock, LW_SHARED);
for (index = 0; index < arrayP->numProcs; index++)
@@ -3029,7 +3040,8 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
GET_VXID_FROM_PGPROC(vxid, *proc);
- if (proc->delayChkpt && VirtualTransactionIdIsValid(vxid))
+ if ((proc->delayChkpt & type) != 0 &&
+ VirtualTransactionIdIsValid(vxid))
{
int i;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 897045ee27..7915cdd484 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -394,7 +394,7 @@ InitProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
if (IsAutoVacuumWorkerProcess())
@@ -576,7 +576,7 @@ InitAuxiliaryProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = false;
MyProc->lwWaitMode = 0;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index a777cb64a1..2799debdaf 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -79,6 +79,10 @@ struct XidCache
*/
#define INVALID_PGPROCNO PG_INT32_MAX
+/* symbols for PGPROC.delayChkpt */
+#define DELAY_CHKPT_START (1<<0)
+#define DELAY_CHKPT_COMPLETE (1<<1)
+
typedef enum
{
PROC_WAIT_STATUS_OK,
@@ -184,7 +188,8 @@ struct PGPROC
pg_atomic_uint64 waitStart; /* time at which wait for lock acquisition
* started */
- bool delayChkpt; /* true if this proc delays checkpoint start */
+ int delayChkpt; /* if this proc delays checkpoint start and/or
+ * completion. */
uint8 statusFlags; /* this backend's status flags, see PROC_*
* above. mirrored in
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index b01fa52139..ec40130466 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -15,11 +15,11 @@
#define PROCARRAY_H
#include "storage/lock.h"
+#include "storage/proc.h"
#include "storage/standby.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
-
extern Size ProcArrayShmemSize(void);
extern void CreateSharedProcArray(void);
extern void ProcArrayAdd(PGPROC *proc);
@@ -59,8 +59,9 @@ extern TransactionId GetOldestActiveTransactionId(void);
extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly);
extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin);
-extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids);
-extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids);
+extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
+extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
+ int nvxids, int type);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
Attachments:
[text/plain] delay_chkpt_cmpl_after_trunc_sucess_2.patch.txt (13.1K, ../../[email protected]/2-delay_chkpt_cmpl_after_trunc_sucess_2.patch.txt)
download | inline diff:
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1fa1..c1b0b48362 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3072,8 +3072,8 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
* crash/basebackup, even though the state of the data directory would
* require it.
*/
- Assert(!MyProc->delayChkpt);
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/* WAL log truncation */
WriteMTruncateXlogRec(newOldestMultiDB,
@@ -3099,7 +3099,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
/* Then offsets */
PerformOffsetsTruncation(oldestMulti, newOldestMulti);
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
LWLockRelease(MultiXactTruncationLock);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20d6c..85c720491b 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -463,7 +463,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
proc->lxid = (LocalTransactionId) xid;
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
- proc->delayChkpt = false;
+ proc->delayChkpt = 0;
proc->statusFlags = 0;
proc->pid = 0;
proc->backendId = InvalidBackendId;
@@ -1109,7 +1109,8 @@ EndPrepare(GlobalTransaction gxact)
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
XLogBeginInsert();
for (record = records.head; record != NULL; record = record->next)
@@ -1152,7 +1153,7 @@ EndPrepare(GlobalTransaction gxact)
* checkpoint starting after this will certainly see the gxact as a
* candidate for fsyncing.
*/
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
/*
* Remember that we have this GlobalTransaction entry locked for us. If
@@ -2198,7 +2199,8 @@ RecordTransactionCommitPrepared(TransactionId xid,
START_CRIT_SECTION();
/* See notes in RecordTransactionCommit */
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/*
* Emit the XLOG commit record. Note that we mark 2PC commits as
@@ -2246,7 +2248,7 @@ RecordTransactionCommitPrepared(TransactionId xid,
TransactionIdCommitTree(xid, nchildren, children);
/* Checkpoint can proceed now */
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 4e6a3df6b8..f033e8940a 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1334,8 +1334,9 @@ RecordTransactionCommit(void)
* This makes checkpoint's determination of which xacts are delayChkpt
* a bit fuzzy, but it doesn't matter.
*/
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
SetCurrentTransactionStopTimestamp();
@@ -1436,7 +1437,7 @@ RecordTransactionCommit(void)
*/
if (markXidCommitted)
{
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
}
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 377afb8732..5f5703bd57 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9065,18 +9065,30 @@ CreateCheckPoint(int flags)
* and we will correctly flush the update below. So we cannot miss any
* xacts we need to wait for.
*/
- vxids = GetVirtualXIDsDelayingChkpt(&nvxids);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_START);
if (nvxids > 0)
{
do
{
pg_usleep(10000L); /* wait for 10 msec */
- } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids));
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_START));
}
pfree(vxids);
CheckPointGuts(checkPoint.redo, flags);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_COMPLETE);
+ if (nvxids > 0)
+ {
+ do
+ {
+ pg_usleep(10000L); /* wait for 10 msec */
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_COMPLETE));
+ }
+ pfree(vxids);
+
/*
* Take a snapshot of running transactions and write this to WAL. This
* allows us to reconstruct the state of running transactions during
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 7052dc245e..1edd1b67ff 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -923,7 +923,7 @@ XLogSaveBufferForHint(Buffer buffer, bool buffer_std)
/*
* Ensure no checkpoint can change our view of RedoRecPtr.
*/
- Assert(MyProc->delayChkpt);
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) != 0);
/*
* Update RedoRecPtr so that we can make the right decision
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index cba7a9ada0..579f23c991 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -325,6 +325,16 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
RelationPreTruncate(rel);
+ /*
+ * If the file truncation fails but the concurrent checkpoint completes
+ * just before that, the next crash recovery can fail due to WAL records
+ * inconsistent with the untruncated pages. To avoid that situation we
+ * delay the checkpoint completion until we confirm the truncation to be
+ * successful.
+ */
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_COMPLETE) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_COMPLETE;
+
/*
* We WAL-log the truncation before actually truncating, which means
* trouble if the truncation fails. If we then crash, the WAL replay
@@ -373,6 +383,8 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
*/
if (need_fsm_vacuum)
FreeSpaceMapVacuumRange(rel, nblocks, InvalidBlockNumber);
+
+ MyProc->delayChkpt &= ~DELAY_CHKPT_COMPLETE;
}
/*
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 561c212092..1c9e971b31 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3803,7 +3803,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
{
XLogRecPtr lsn = InvalidXLogRecPtr;
bool dirtied = false;
- bool delayChkpt = false;
+ int delayChkptMask = ~0;
uint32 buf_state;
/*
@@ -3853,7 +3853,9 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
* essential that CreateCheckpoint waits for virtual transactions
* rather than full transactionids.
*/
- MyProc->delayChkpt = delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
+ delayChkptMask = ~DELAY_CHKPT_START;
lsn = XLogSaveBufferForHint(buffer, buffer_std);
}
@@ -3885,8 +3887,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
buf_state |= BM_DIRTY | BM_JUST_DIRTIED;
UnlockBufHdr(bufHdr, buf_state);
- if (delayChkpt)
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= delayChkptMask;
if (dirtied)
{
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 4fc6ffb917..3e6759886a 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -655,7 +655,10 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -694,7 +697,10 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
proc->xid = InvalidTransactionId;
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -2955,7 +2961,8 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* delaying checkpoint because they have critical actions in progress.
*
* Constructs an array of VXIDs of transactions that are currently in commit
- * critical sections, as shown by having delayChkpt set in their PGPROC.
+ * critical sections, as shown by having delayChkpt set to the specified value
+ * in their PGPROC.
*
* Returns a palloc'd array that should be freed by the caller.
* *nvxids is the number of valid entries.
@@ -2969,13 +2976,15 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* for clearing of delayChkpt to propagate is unimportant for correctness.
*/
VirtualTransactionId *
-GetVirtualXIDsDelayingChkpt(int *nvxids)
+GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
{
VirtualTransactionId *vxids;
ProcArrayStruct *arrayP = procArray;
int count = 0;
int index;
+ Assert(type != 0);
+
/* allocate what's certainly enough result space */
vxids = (VirtualTransactionId *)
palloc(sizeof(VirtualTransactionId) * arrayP->maxProcs);
@@ -2987,7 +2996,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];
- if (proc->delayChkpt)
+ if ((proc->delayChkpt & type) != 0)
{
VirtualTransactionId vxid;
@@ -3013,12 +3022,14 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
* those numbers should be small enough for it not to be a problem.
*/
bool
-HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
+HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
{
bool result = false;
ProcArrayStruct *arrayP = procArray;
int index;
+ Assert(type != 0);
+
LWLockAcquire(ProcArrayLock, LW_SHARED);
for (index = 0; index < arrayP->numProcs; index++)
@@ -3029,7 +3040,8 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
GET_VXID_FROM_PGPROC(vxid, *proc);
- if (proc->delayChkpt && VirtualTransactionIdIsValid(vxid))
+ if ((proc->delayChkpt & type) != 0 &&
+ VirtualTransactionIdIsValid(vxid))
{
int i;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 897045ee27..7915cdd484 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -394,7 +394,7 @@ InitProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
if (IsAutoVacuumWorkerProcess())
@@ -576,7 +576,7 @@ InitAuxiliaryProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = false;
MyProc->lwWaitMode = 0;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index a777cb64a1..2799debdaf 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -79,6 +79,10 @@ struct XidCache
*/
#define INVALID_PGPROCNO PG_INT32_MAX
+/* symbols for PGPROC.delayChkpt */
+#define DELAY_CHKPT_START (1<<0)
+#define DELAY_CHKPT_COMPLETE (1<<1)
+
typedef enum
{
PROC_WAIT_STATUS_OK,
@@ -184,7 +188,8 @@ struct PGPROC
pg_atomic_uint64 waitStart; /* time at which wait for lock acquisition
* started */
- bool delayChkpt; /* true if this proc delays checkpoint start */
+ int delayChkpt; /* if this proc delays checkpoint start and/or
+ * completion. */
uint8 statusFlags; /* this backend's status flags, see PROC_*
* above. mirrored in
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index b01fa52139..ec40130466 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -15,11 +15,11 @@
#define PROCARRAY_H
#include "storage/lock.h"
+#include "storage/proc.h"
#include "storage/standby.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
-
extern Size ProcArrayShmemSize(void);
extern void CreateSharedProcArray(void);
extern void ProcArrayAdd(PGPROC *proc);
@@ -59,8 +59,9 @@ extern TransactionId GetOldestActiveTransactionId(void);
extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly);
extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin);
-extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids);
-extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids);
+extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
+extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
+ int nvxids, int type);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-08-10 18:14 Robert Haas <[email protected]>
parent: Kyotaro Horiguchi <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Robert Haas @ 2021-08-10 18:14 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Thu, Mar 4, 2021 at 10:01 PM Kyotaro Horiguchi
<[email protected]> wrote:
> The patch assumed that CHKPT_START/COMPLETE barrier are exclusively
> used each other, but MarkBufferDirtyHint which delays checkpoint start
> is called in RelationTruncate while delaying checkpoint completion.
> That is not a strange nor harmful behavior. I changed delayChkpt to a
> bitmap integer from an enum so that both barrier are separately
> triggered.
>
> I'm not sure this is the way to go here, though. This fixes the issue
> of a crash during RelationTruncate, but the issue of smgrtruncate
> failure during RelationTruncate still remains (unless we treat that
> failure as PANIC?).
I like this patch. As I understand it, we're currently cheating by
allowing checkpoints to complete without necessarily flushing all of
the pages that were dirty at the time we fixed the redo pointer out to
disk. We think this is OK because we know that those pages are going
to get truncated away, but it's not really OK because when the system
starts up, it has to replay WAL starting from the checkpoint's redo
pointer, but the state of the page is not the same as it was at the
time when the redo pointer was the end of WAL, so redo fails. In the
case described in
http://postgr.es/m/BYAPR06MB63739B2692DC6DBB3C5F186CABDA0@BYAPR06MB6373.namprd06.prod.outlook.com
modifications are made to the page before the redo pointer is fixed
and those changes never make it to disk, but the truncation also never
makes it to the disk either. With this patch, that can't happen,
because no checkpoint can intervene between when we (1) decide we're
not going to bother writing those dirty pages and (2) actually
truncate them away. So either the pages will get written as part of
the checkpoint, or else they'll be gone before the checkpoint
completes. In the latter case, I suppose redo that would have modified
those pages will just be skipped, thus dodging the problem.
In RelationTruncate, I suggest that we ought to clear the
delay-checkpoint flag before rather than after calling
FreeSpaceMapVacuumRange. Since the free space map is not fully
WAL-logged, anything we're doing there should be non-critical. Also, I
think it might be better if MarkBufferDirtyHint stays closer to the
existing coding and just uses a Boolean and an if-test to decide
whether to clear the bit, instead of inventing a new mechanism. I
don't really see anything wrong with the new mechanism, but I think
it's better to keep the patch minimal.
As you say, this doesn't fix the problem that truncation might fail.
But as Andres and Sawada-san said, the solution to that is to get rid
of the comments saying that it's OK for truncation to fail and make it
a PANIC. However, I don't think that change needs to be part of this
patch. Even if we do that, we still need to do this. And even if we do
this, we still need to do that.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-09-24 19:37 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2021-09-24 19:37 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> I like this patch.
I think the basic idea is about right, but I'm not happy with the
three-way delayChkpt business; that seems too cute by three-quarters.
I think two independent boolean flags, one saying "I'm preventing
checkpoint start" and one saying "I'm preventing checkpoint completion",
would be much less confusing and also more future-proof. Who's to say
that we won't ever need both states to be set in the same process?
I also dislike the fact that the patch has made procarray.h depend
on proc.h ... maybe I'm wrong, but I thought that there was a reason
for keeping those independent, if indeed this hasn't actually resulted
in a circular-includes situation. If we avoid inventing that enum type
then there's no need for that. If we do need an enum, maybe it could
be put in some already-common prerequisite header.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-09-24 20:08 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Robert Haas @ 2021-09-24 20:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Sep 24, 2021 at 3:42 PM Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
> > I like this patch.
>
> I think the basic idea is about right, but I'm not happy with the
> three-way delayChkpt business; that seems too cute by three-quarters.
> I think two independent boolean flags, one saying "I'm preventing
> checkpoint start" and one saying "I'm preventing checkpoint completion",
> would be much less confusing and also more future-proof. Who's to say
> that we won't ever need both states to be set in the same process?
Nobody, but the version of the patch that I was looking at uses a
separate bit for each one:
+/* symbols for PGPROC.delayChkpt */
+#define DELAY_CHKPT_START (1<<0)
+#define DELAY_CHKPT_COMPLETE (1<<1)
One could instead use separate Booleans, but there doesn't seem to be
anything three-way about this?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-09-24 20:22 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Tom Lane @ 2021-09-24 20:22 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> On Fri, Sep 24, 2021 at 3:42 PM Tom Lane <[email protected]> wrote:
>> I think the basic idea is about right, but I'm not happy with the
>> three-way delayChkpt business; that seems too cute by three-quarters.
> Nobody, but the version of the patch that I was looking at uses a
> separate bit for each one:
> +/* symbols for PGPROC.delayChkpt */
> +#define DELAY_CHKPT_START (1<<0)
> +#define DELAY_CHKPT_COMPLETE (1<<1)
Hm, that's not in the patch version that the CF app claims to be
latest [1]. It does this:
+/* type for PGPROC.delayChkpt */
+typedef enum DelayChkptType
+{
+ DELAY_CHKPT_NONE = 0,
+ DELAY_CHKPT_START,
+ DELAY_CHKPT_COMPLETE
+} DelayChkptType;
which seems like a distinct disimprovement over what you're quoting.
regards, tom lane
[1] https://www.postgresql.org/message-id/[email protected]
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-09-27 08:28 Kyotaro Horiguchi <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Kyotaro Horiguchi @ 2021-09-27 08:28 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]
Thaks for looking this, Robert and Tom.
At Fri, 24 Sep 2021 16:22:28 -0400, Tom Lane <[email protected]> wrote in
> Robert Haas <[email protected]> writes:
> > On Fri, Sep 24, 2021 at 3:42 PM Tom Lane <[email protected]> wrote:
> >> I think the basic idea is about right, but I'm not happy with the
> >> three-way delayChkpt business; that seems too cute by three-quarters.
>
> > Nobody, but the version of the patch that I was looking at uses a
> > separate bit for each one:
>
> > +/* symbols for PGPROC.delayChkpt */
> > +#define DELAY_CHKPT_START (1<<0)
> > +#define DELAY_CHKPT_COMPLETE (1<<1)
>
> Hm, that's not in the patch version that the CF app claims to be
> latest [1]. It does this:
>
> +/* type for PGPROC.delayChkpt */
> +typedef enum DelayChkptType
> +{
> + DELAY_CHKPT_NONE = 0,
> + DELAY_CHKPT_START,
> + DELAY_CHKPT_COMPLETE
> +} DelayChkptType;
>
> which seems like a distinct disimprovement over what you're quoting.
Yeah, that is because the latest patch is not attached as *.patch/diff
but *.txt. I didn't name it as *.patch in order to avoid noise patch
in that thread although it was too late. On the contrary that seems to
have lead in another trouble..
Tom's concern is right. Actually both the two events can happen
simultaneously but the latest *.patch.txt treats that case as Robert
said.
One advantage of having the two flags as one bitmap integer is it
slightly simplifies the logic in GetVirtualXIDsDelayingChkpt and
HaveVirtualXIDsDelayingChkpt. On the other hand it very slightly
complexifies how to set/reset the flags.
GetVirtualXIDsDelayingChkpt:
+ if ((proc->delayChkpt & type) != 0)
vs
+ if (delayStart)
+ delayflag = proc->delayChkptStart;
+ else
+ delayflag = proc->delayChkptEnd;
+ if (delayflag != 0)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2021-09-27 08:30 Kyotaro Horiguchi <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Kyotaro Horiguchi @ 2021-09-27 08:30 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]
Thank you for the comments! (Sorry for the late resopnse.)
At Tue, 10 Aug 2021 14:14:05 -0400, Robert Haas <[email protected]> wrote in
> On Thu, Mar 4, 2021 at 10:01 PM Kyotaro Horiguchi
> <[email protected]> wrote:
> > The patch assumed that CHKPT_START/COMPLETE barrier are exclusively
> > used each other, but MarkBufferDirtyHint which delays checkpoint start
> > is called in RelationTruncate while delaying checkpoint completion.
> > That is not a strange nor harmful behavior. I changed delayChkpt to a
> > bitmap integer from an enum so that both barrier are separately
> > triggered.
> >
> > I'm not sure this is the way to go here, though. This fixes the issue
> > of a crash during RelationTruncate, but the issue of smgrtruncate
> > failure during RelationTruncate still remains (unless we treat that
> > failure as PANIC?).
>
> I like this patch. As I understand it, we're currently cheating by
> allowing checkpoints to complete without necessarily flushing all of
> the pages that were dirty at the time we fixed the redo pointer out to
> disk. We think this is OK because we know that those pages are going
> to get truncated away, but it's not really OK because when the system
> starts up, it has to replay WAL starting from the checkpoint's redo
> pointer, but the state of the page is not the same as it was at the
> time when the redo pointer was the end of WAL, so redo fails. In the
> case described in
> http://postgr.es/m/BYAPR06MB63739B2692DC6DBB3C5F186CABDA0@BYAPR06MB6373.namprd06.prod.outlook.com
> modifications are made to the page before the redo pointer is fixed
> and those changes never make it to disk, but the truncation also never
> makes it to the disk either. With this patch, that can't happen,
> because no checkpoint can intervene between when we (1) decide we're
> not going to bother writing those dirty pages and (2) actually
> truncate them away. So either the pages will get written as part of
> the checkpoint, or else they'll be gone before the checkpoint
> completes. In the latter case, I suppose redo that would have modified
> those pages will just be skipped, thus dodging the problem.
I think your understanding is right.
> In RelationTruncate, I suggest that we ought to clear the
> delay-checkpoint flag before rather than after calling
> FreeSpaceMapVacuumRange. Since the free space map is not fully
> WAL-logged, anything we're doing there should be non-critical. Also, I
Agreed and fixed.
> think it might be better if MarkBufferDirtyHint stays closer to the
> existing coding and just uses a Boolean and an if-test to decide
> whether to clear the bit, instead of inventing a new mechanism. I
> don't really see anything wrong with the new mechanism, but I think
> it's better to keep the patch minimal.
Yeah, that was a a kind of silly. Fixed.
> As you say, this doesn't fix the problem that truncation might fail.
> But as Andres and Sawada-san said, the solution to that is to get rid
> of the comments saying that it's OK for truncation to fail and make it
> a PANIC. However, I don't think that change needs to be part of this
> patch. Even if we do that, we still need to do this. And even if we do
> this, we still need to do that.
Ok. Addition to the aboves, I rewrote the comment in RelatinoTruncate.
+ * Delay the concurrent checkpoint's completion until this truncation
+ * successfully completes, so that we don't establish a redo-point between
+ * buffer deletion and file-truncate. Otherwise we can leave inconsistent
+ * file content against the WAL records after the REDO position and future
+ * recovery fails.
However, a problem for me for now is that I cannot reproduce the
problem.
To avoid further confusion, the attached is named as *.patch.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments:
[text/x-patch] delay_chkpt_cmpl_after_trunc_sucess_v3.patch (13.1K, ../../[email protected]/2-delay_chkpt_cmpl_after_trunc_sucess_v3.patch)
download | inline diff:
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index e6c70ed0bc..17357179e3 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3075,8 +3075,8 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
* crash/basebackup, even though the state of the data directory would
* require it.
*/
- Assert(!MyProc->delayChkpt);
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/* WAL log truncation */
WriteMTruncateXlogRec(newOldestMultiDB,
@@ -3102,7 +3102,7 @@ TruncateMultiXact(MultiXactId newOldestMulti, Oid newOldestMultiDB)
/* Then offsets */
PerformOffsetsTruncation(oldestMulti, newOldestMulti);
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
LWLockRelease(MultiXactTruncationLock);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 2156de187c..b7dc84d6e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -463,7 +463,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
proc->lxid = (LocalTransactionId) xid;
proc->xid = xid;
Assert(proc->xmin == InvalidTransactionId);
- proc->delayChkpt = false;
+ proc->delayChkpt = 0;
proc->statusFlags = 0;
proc->pid = 0;
proc->backendId = InvalidBackendId;
@@ -1109,7 +1109,8 @@ EndPrepare(GlobalTransaction gxact)
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
XLogBeginInsert();
for (record = records.head; record != NULL; record = record->next)
@@ -1152,7 +1153,7 @@ EndPrepare(GlobalTransaction gxact)
* checkpoint starting after this will certainly see the gxact as a
* candidate for fsyncing.
*/
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
/*
* Remember that we have this GlobalTransaction entry locked for us. If
@@ -2215,7 +2216,8 @@ RecordTransactionCommitPrepared(TransactionId xid,
START_CRIT_SECTION();
/* See notes in RecordTransactionCommit */
- MyProc->delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
/*
* Emit the XLOG commit record. Note that we mark 2PC commits as
@@ -2263,7 +2265,7 @@ RecordTransactionCommitPrepared(TransactionId xid,
TransactionIdCommitTree(xid, nchildren, children);
/* Checkpoint can proceed now */
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 6597ec45a9..4a1a0c3c1f 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -1334,8 +1334,9 @@ RecordTransactionCommit(void)
* This makes checkpoint's determination of which xacts are delayChkpt
* a bit fuzzy, but it doesn't matter.
*/
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
START_CRIT_SECTION();
- MyProc->delayChkpt = true;
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
SetCurrentTransactionStopTimestamp();
@@ -1436,7 +1437,7 @@ RecordTransactionCommit(void)
*/
if (markXidCommitted)
{
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
END_CRIT_SECTION();
}
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index e51a7a749d..a4d564323a 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -9153,18 +9153,30 @@ CreateCheckPoint(int flags)
* and we will correctly flush the update below. So we cannot miss any
* xacts we need to wait for.
*/
- vxids = GetVirtualXIDsDelayingChkpt(&nvxids);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_START);
if (nvxids > 0)
{
do
{
pg_usleep(10000L); /* wait for 10 msec */
- } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids));
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_START));
}
pfree(vxids);
CheckPointGuts(checkPoint.redo, flags);
+ vxids = GetVirtualXIDsDelayingChkpt(&nvxids, DELAY_CHKPT_COMPLETE);
+ if (0 && nvxids > 0)
+ {
+ do
+ {
+ pg_usleep(10000L); /* wait for 10 msec */
+ } while (HaveVirtualXIDsDelayingChkpt(vxids, nvxids,
+ DELAY_CHKPT_COMPLETE));
+ }
+ pfree(vxids);
+
/*
* Take a snapshot of running transactions and write this to WAL. This
* allows us to reconstruct the state of running transactions during
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index b492c656d7..f7a1f981d5 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -978,7 +978,7 @@ XLogSaveBufferForHint(Buffer buffer, bool buffer_std)
/*
* Ensure no checkpoint can change our view of RedoRecPtr.
*/
- Assert(MyProc->delayChkpt);
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) != 0);
/*
* Update RedoRecPtr so that we can make the right decision
diff --git a/src/backend/catalog/storage.c b/src/backend/catalog/storage.c
index c5ad28d71f..be9c0e107f 100644
--- a/src/backend/catalog/storage.c
+++ b/src/backend/catalog/storage.c
@@ -325,6 +325,16 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
RelationPreTruncate(rel);
+ /*
+ * Delay the concurrent checkpoint's completion until this truncation
+ * successfully completes, so that we don't establish a redo-point between
+ * buffer deletion and file-truncate. Otherwise we can leave inconsistent
+ * file content against the WAL records after the REDO position and future
+ * recovery fails.
+ */
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_COMPLETE) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_COMPLETE;
+
/*
* We WAL-log the truncation before actually truncating, which means
* trouble if the truncation fails. If we then crash, the WAL replay
@@ -366,6 +376,10 @@ RelationTruncate(Relation rel, BlockNumber nblocks)
/* Do the real work to truncate relation forks */
smgrtruncate(RelationGetSmgr(rel), forks, nforks, blocks);
+
+ /* FSM is not WAL-logged, finish the critical section here. */
+ MyProc->delayChkpt &= ~DELAY_CHKPT_COMPLETE;
+
/*
* Update upper-level FSM pages to account for the truncation. This is
* important because the just-truncated pages were likely marked as
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index e88e4e918b..c277dc3e1e 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -3921,7 +3921,9 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
* essential that CreateCheckpoint waits for virtual transactions
* rather than full transactionids.
*/
- MyProc->delayChkpt = delayChkpt = true;
+ Assert((MyProc->delayChkpt & DELAY_CHKPT_START) == 0);
+ MyProc->delayChkpt |= DELAY_CHKPT_START;
+ delayChkpt = true;
lsn = XLogSaveBufferForHint(buffer, buffer_std);
}
@@ -3954,7 +3956,7 @@ MarkBufferDirtyHint(Buffer buffer, bool buffer_std)
UnlockBufHdr(bufHdr, buf_state);
if (delayChkpt)
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt &= ~DELAY_CHKPT_START;
if (dirtied)
{
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index bd3c7a47fe..1bc4ea15e9 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -689,7 +689,10 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -728,7 +731,10 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
proc->xid = InvalidTransactionId;
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
- proc->delayChkpt = false; /* be sure this is cleared in abort */
+
+ /* be sure this is cleared in abort */
+ proc->delayChkpt = 0;
+
proc->recoveryConflictPending = false;
/* must be cleared with xid/xmin: */
@@ -3026,7 +3032,8 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* delaying checkpoint because they have critical actions in progress.
*
* Constructs an array of VXIDs of transactions that are currently in commit
- * critical sections, as shown by having delayChkpt set in their PGPROC.
+ * critical sections, as shown by having delayChkpt set to the specified value
+ * in their PGPROC.
*
* Returns a palloc'd array that should be freed by the caller.
* *nvxids is the number of valid entries.
@@ -3040,13 +3047,15 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly)
* for clearing of delayChkpt to propagate is unimportant for correctness.
*/
VirtualTransactionId *
-GetVirtualXIDsDelayingChkpt(int *nvxids)
+GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
{
VirtualTransactionId *vxids;
ProcArrayStruct *arrayP = procArray;
int count = 0;
int index;
+ Assert(type != 0);
+
/* allocate what's certainly enough result space */
vxids = (VirtualTransactionId *)
palloc(sizeof(VirtualTransactionId) * arrayP->maxProcs);
@@ -3058,7 +3067,7 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];
- if (proc->delayChkpt)
+ if ((proc->delayChkpt & type) != 0)
{
VirtualTransactionId vxid;
@@ -3084,12 +3093,14 @@ GetVirtualXIDsDelayingChkpt(int *nvxids)
* those numbers should be small enough for it not to be a problem.
*/
bool
-HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
+HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
{
bool result = false;
ProcArrayStruct *arrayP = procArray;
int index;
+ Assert(type != 0);
+
LWLockAcquire(ProcArrayLock, LW_SHARED);
for (index = 0; index < arrayP->numProcs; index++)
@@ -3100,7 +3111,8 @@ HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
GET_VXID_FROM_PGPROC(vxid, *proc);
- if (proc->delayChkpt && VirtualTransactionIdIsValid(vxid))
+ if ((proc->delayChkpt & type) != 0 &&
+ VirtualTransactionIdIsValid(vxid))
{
int i;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index b7d9da0aa9..95fdf990e7 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -394,7 +394,7 @@ InitProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
/* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */
if (IsAutoVacuumWorkerProcess())
@@ -579,7 +579,7 @@ InitAuxiliaryProcess(void)
MyProc->roleId = InvalidOid;
MyProc->tempNamespaceId = InvalidOid;
MyProc->isBackgroundWorker = IsBackgroundWorker;
- MyProc->delayChkpt = false;
+ MyProc->delayChkpt = 0;
MyProc->statusFlags = 0;
MyProc->lwWaiting = false;
MyProc->lwWaitMode = 0;
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index be67d8a861..b9be2454c5 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -79,6 +79,10 @@ struct XidCache
*/
#define INVALID_PGPROCNO PG_INT32_MAX
+/* symbols for PGPROC.delayChkpt */
+#define DELAY_CHKPT_START (1<<0)
+#define DELAY_CHKPT_COMPLETE (1<<1)
+
typedef enum
{
PROC_WAIT_STATUS_OK,
@@ -184,7 +188,8 @@ struct PGPROC
pg_atomic_uint64 waitStart; /* time at which wait for lock acquisition
* started */
- bool delayChkpt; /* true if this proc delays checkpoint start */
+ int delayChkpt; /* if this proc delays checkpoint start and/or
+ * completion. */
uint8 statusFlags; /* this backend's status flags, see PROC_*
* above. mirrored in
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index b01fa52139..ec40130466 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -15,11 +15,11 @@
#define PROCARRAY_H
#include "storage/lock.h"
+#include "storage/proc.h"
#include "storage/standby.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
-
extern Size ProcArrayShmemSize(void);
extern void CreateSharedProcArray(void);
extern void ProcArrayAdd(PGPROC *proc);
@@ -59,8 +59,9 @@ extern TransactionId GetOldestActiveTransactionId(void);
extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly);
extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin);
-extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids);
-extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids);
+extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids, int type);
+extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids,
+ int nvxids, int type);
extern PGPROC *BackendPidGetProc(int pid);
extern PGPROC *BackendPidGetProcWithLock(int pid);
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-24 19:33 Robert Haas <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Robert Haas @ 2022-03-24 19:33 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Thu, Mar 17, 2022 at 9:21 PM Kyotaro Horiguchi
<[email protected]> wrote:
> Finally, no two of from 10 to 14 doesn't accept the same patch.
>
> As a cross-version check, I compared all combinations of the patches
> for two adjacent versions and confirmed that no hunks are lost.
>
> All versions pass check world.
Thanks, committed.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-24 22:04 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-24 22:04 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> Thanks, committed.
Some of the buildfarm is seeing failures in the pg_checksums test.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 00:37 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Robert Haas @ 2022-03-25 00:37 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Thu, Mar 24, 2022 at 6:04 PM Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
> > Thanks, committed.
>
> Some of the buildfarm is seeing failures in the pg_checksums test.
Hmm. So the tests seem to be failing because 002_actions.pl stops the
database cluster, runs pg_checksums (which passes), writes some zero
bytes over the line pointer array of the first block of pg_class, and
then runs pg_checksums again. In the failing buildfarm runs,
pg_checksums fails to detect the corruption: the second run succeeds,
while pg_checksums expects it to fail. That's pretty curious, because
if the database cluster is stopped, and things are OK at that point,
then how could a server bug of any kind cause a Perl script to be
unable to corrupt a file on disk?
A possible clue is that I also see a few machines failing in
recoveryCheck. And the code that is failing there looks like this:
# We've seen occasional cases where multiple walsender pids are active. An
# immediate shutdown may hide evidence of a locking bug. So if multiple
# walsenders are observed, shut down in fast mode, and collect some more
# information.
if (not like($senderpid, qr/^[0-9]+$/, "have walsender pid $senderpid"))
{
my ($stdout, $stderr);
$node_primary3->psql('postgres',
"\\a\\t\nSELECT * FROM pg_stat_activity",
stdout => \$stdout, stderr => \$stderr);
diag $stdout, $stderr;
$node_primary3->stop('fast');
$node_standby3->stop('fast');
die "could not determine walsender pid, can't continue";
}
And the failure looks like this:
# Failed test 'have walsender pid 1047504
# 1047472'
# at t/019_replslot_limit.pl line 343.
That sure looks like there are multiple walsender PIDs active, and the
pg_stat_activity output confirms it. 1047504 is running
START_REPLICATION SLOT "rep3" 0/700000 TIMELINE 1 and 1047472 is
running START_REPLICATION SLOT "pg_basebackup_1047472" 0/600000
TIMELINE 1.
Both of these failures could possibly be explained by some failure of
things to shut down properly, but it's not the same things. In the
first case, the database server would have had to still be running
after we run $node->stop, and it would have had to overwrite the bad
contents of pg_class with some good contents. In the second case, the
cluster's supposed to still be running, but the backends that were
creating those replication slots should have exited sooner.
I've been running the pg_checksums test in a loop here for a bit now
in the hopes of being able to reproduce the failure, but it doesn't
seem to want to fail here. And I've also looked over the commit and I
can't quite see how it would cause a process, or the cluster, to fail
to shutdown, unless perhaps it's the checkpointer that gets stuck, but
that doesn't really seem to match the symptoms.
Any ideas?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 00:39 Robert Haas <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Robert Haas @ 2022-03-25 00:39 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Thu, Mar 24, 2022 at 8:37 PM Robert Haas <[email protected]> wrote:
> Any ideas?
And ... right after hitting send, I see that the recovery check
failures are under separate troubleshooting and thus probably
unrelated. But that leaves me even more confused. How can a change to
only the server code cause a client utility to fail to detect
corruption that is being created by Perl while the server is stopped?
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 00:45 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 00:45 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> And ... right after hitting send, I see that the recovery check
> failures are under separate troubleshooting and thus probably
> unrelated.
Yeah, we've been chasing those for months.
> But that leaves me even more confused. How can a change to
> only the server code cause a client utility to fail to detect
> corruption that is being created by Perl while the server is stopped?
Hmm, I'd supposed that the failing test cases were new as of 412ad7a55.
Now I see they're not, which indeed puts quite a different spin on
things. Your thought about maybe the server isn't shut down yet is
interesting --- did 412ad7a55 touch anything about the shutdown
sequence?
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 01:08 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Robert Haas @ 2022-03-25 01:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Thu, Mar 24, 2022 at 8:45 PM Tom Lane <[email protected]> wrote:
> Hmm, I'd supposed that the failing test cases were new as of 412ad7a55.
> Now I see they're not, which indeed puts quite a different spin on
> things. Your thought about maybe the server isn't shut down yet is
> interesting --- did 412ad7a55 touch anything about the shutdown
> sequence?
I hate to say "no" because the evidence suggests that the answer might
be "yes" -- but it definitely isn't intending to change anything about
the shutdown sequence. It just introduces a mechanism to backends to
force the checkpointer to delay writing the checkpoint record.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 01:22 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Tom Lane @ 2022-03-25 01:22 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; Andres Freund <[email protected]>; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> I hate to say "no" because the evidence suggests that the answer might
> be "yes" -- but it definitely isn't intending to change anything about
> the shutdown sequence. It just introduces a mechanism to backends to
> force the checkpointer to delay writing the checkpoint record.
Wait a minute, I think we may be barking up the wrong tree.
The three commits that serinus saw as new in its first failure were
ce95c54376 Thu Mar 24 20:33:13 2022 UTC Fix pg_statio_all_tables view for multiple TOAST indexes.
7dac61402e Thu Mar 24 19:51:40 2022 UTC Remove unused module imports from TAP tests
412ad7a556 Thu Mar 24 18:52:28 2022 UTC Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.
I failed to look closely at dragonet, but I now see that its
first failure saw
ce95c54376 Thu Mar 24 20:33:13 2022 UTC Fix pg_statio_all_tables view for multiple TOAST indexes.
7dac61402e Thu Mar 24 19:51:40 2022 UTC Remove unused module imports from TAP tests
serinus is 0-for-3 since then, and dragonet 0-for-4, so we can be pretty
confident that the failure is repeatable for them. That means that the
culprit must be ce95c54376 or 7dac61402e, not anything nearby such as
412ad7a556.
It's *really* hard to see how the pg_statio_all_tables change could
have affected this. So that leaves 7dac61402e, which did this to
the test script that's failing:
use strict;
use warnings;
-use Config;
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
Discuss.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 01:23 Andres Freund <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Andres Freund @ 2022-03-25 01:23 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 20:39:27 -0400, Robert Haas wrote:
> But that leaves me even more confused. How can a change to only the server
> code cause a client utility to fail to detect corruption that is being
> created by Perl while the server is stopped?
I guess it could somehow cause the first page to be all zeroes, in which case
overwriting it with more zeroes wouldn't cause a problem that pg_checksums can
see? But I have a somewhat more realistic idea:
I'm suspicious of pg_checksums --filenode. If I understand correctly
--filenode scans the data directory, including all tablespaces, for a file
matching that filenode. If we somehow end up with a leftover file in the pre
ALTER TABLE SET TABLESPACE location, it'd not notice that there *also* is a
file in a different place?
Perhaps the --filenode mode should print out the file location...
Randomly noticed: The test fetches the block size without doing anything with
it afaics.
Andres
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 01:59 Tom Lane <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 01:59 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
I wrote:
> ... So that leaves 7dac61402e, which did this to
> the test script that's failing:
> use strict;
> use warnings;
> -use Config;
> use PostgreSQL::Test::Cluster;
> use PostgreSQL::Test::Utils;
> Discuss.
Another thing that seems quite baffling, but is becoming clearer by
the hour, is that only serinus and dragonet are seeing this failure.
How is that? They're not very similarly configured --- one is gcc,
one clang, and one uses jit and one doesn't. They do share the same
perl version, 5.34.0; but so do twenty-three other animals, many of
which have reported in cleanly. I'm at a loss to explain that.
Andres, can you think of anything that's peculiar to those two
animals?
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 02:14 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 02:14 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 21:22:38 -0400, Tom Lane wrote:
> serinus is 0-for-3 since then, and dragonet 0-for-4, so we can be pretty
> confident that the failure is repeatable for them.
That's weird. They run on the same host, but otherwise they have very little
in common. There's plenty other animals running on the same machine that
didn't report errors.
I copied serinus' configuration, ran the tests repeatedly, without reproducing
the failure so far. Odd.
Combined with the replslot failure I'd be prepared to think the machine has
issues, except that the replslot thing triggered on other machines too.
I looked through logs on the machine without finding anything indicating
something odd.
I turned on keep_error_builds for serinus. Hopefully that'll leave us with
on-disk files to inspect.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 02:20 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 02:20 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 21:59:08 -0400, Tom Lane wrote:
> Another thing that seems quite baffling, but is becoming clearer by
> the hour, is that only serinus and dragonet are seeing this failure.
> How is that? They're not very similarly configured --- one is gcc,
> one clang, and one uses jit and one doesn't. They do share the same
> perl version, 5.34.0; but so do twenty-three other animals, many of
> which have reported in cleanly. I'm at a loss to explain that.
> Andres, can you think of anything that's peculiar to those two
> animals?
No, I'm quite baffled myself. As I noted in an email I just sent, before
reading this one, I can't explain it, and at least in simple attempts, can't
reproduce it either. And there are animals much closer to each other than
those two...
I forced a run while writing the other email, with keep_error_whatnot, and I
just saw it failing... Looking whether there's anything interesting to glean.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 02:23 Thomas Munro <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Thomas Munro @ 2022-03-25 02:23 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 3:14 PM Andres Freund <[email protected]> wrote:
> On 2022-03-24 21:22:38 -0400, Tom Lane wrote:
> > serinus is 0-for-3 since then, and dragonet 0-for-4, so we can be pretty
> > confident that the failure is repeatable for them.
>
> That's weird. They run on the same host, but otherwise they have very little
> in common. There's plenty other animals running on the same machine that
> didn't report errors.
One random thing I've noticed about serinus is that it seems to drop
UDP packets more than others, but dragonet apparently doesn't:
tmunro=> select animal, count(*) from run where result = 'FAILURE' and
'stats' = any(fail_tests) and snapshot > now() - interval '3 month'
group by 1 order by 2 desc;
animal | count
--------------+-------
serinus | 14
flaviventris | 6
mandrill | 2
bonito | 1
seawasp | 1
crake | 1
sungazer | 1
(7 rows)
Example: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=serinus&dt=2022-03-24%2001:00:14
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 02:35 Andres Freund <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 02:35 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-25 15:23:24 +1300, Thomas Munro wrote:
> One random thing I've noticed about serinus is that it seems to drop
> UDP packets more than others, but dragonet apparently doesn't:
Serinus is built with optimization. Which I guess could lead to other backends
reporting stats more quickly? And of course could lead to running more often
(due to finishing before the next cron invocation). I think I've also
configured my animals to run more often than many other owners.
So I'm not sure how much can be gleaned from raw "failure counts" without
taking the number of runs into account as well?
- Andres
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 02:43 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 02:43 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 19:20:10 -0700, Andres Freund wrote:
> I forced a run while writing the other email, with keep_error_whatnot, and I
> just saw it failing... Looking whether there's anything interesting to glean.
Unfortunately the test drops the table and it doesn't report the filepath of
the failure. So I haven't learned much from the data dir so far.
I still don't see a failure when running the tests in a separate source
tree. Can't explain that. Going to try to get closer to the buildfarm script
run - it'd be a whole lot easier to be able to edit the source of the test and
reproduce...
Just to be sure I'm going to clean out serinus' ccache dir and rerun. I'll
leave dragonet's alone for now.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 03:02 Thomas Munro <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Thomas Munro @ 2022-03-25 03:02 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 3:35 PM Andres Freund <[email protected]> wrote:
> So I'm not sure how much can be gleaned from raw "failure counts" without
> taking the number of runs into account as well?
Ah, right, it does indeed hold the record for most runs in 3 months,
and taking runs into account its "stats" failure rate is clustered
with mandrill and seawasp. Anyway, clearly not relevant because
dragonet doesn't even show up in the list.
animal | runs | stats_test_fail_fraction
---------------+------+--------------------------
mandrill | 158 | 0.0126582278481013
seawasp | 85 | 0.0117647058823529
serinus | 1299 | 0.0107775211701309
sungazer | 174 | 0.00574712643678161
flaviventris | 1292 | 0.00464396284829721
bonito | 313 | 0.00319488817891374
crake | 743 | 0.00134589502018843
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 03:43 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 03:43 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 19:43:02 -0700, Andres Freund wrote:
> Just to be sure I'm going to clean out serinus' ccache dir and rerun. I'll
> leave dragonet's alone for now.
Turns out they had the same dir. But it didn't help.
I haven't yet figured out why, but I now *am* able to reproduce the problem in
the buildfarm built tree. Wonder if there's a path length issue or such
somewhere?
Either way, I can now manipulate the tests and still repro. I made the test
abort after the first failure.
hexedit shows that the file is modified, as we'd expect:
00000000 00 00 00 00 C0 01 5B 01 16 7D 00 00 A0 03 C0 03 00 20 04 20 00 00 00 00 00 00 00 00 00 00 00 00 ......[..}....... . ............
00000020 00 9F 38 00 80 9F 38 00 60 9F 38 00 40 9F 38 00 20 9F 38 00 00 9F 38 00 E0 9E 38 00 C0 9E 38 00 ..8...8.`[email protected]. .8...8...8...8.
And we are checking the right file:
bf@andres-postgres-edb-buildfarm-v1:~/build/buildfarm-serinus/HEAD/pgsql.build$ tmp_install/home/bf/build/buildfarm-serinus/HEAD/inst/bin/pg_checksums --check -D /home/bf/build/buildfarm-serinus/HEAD/pgsql.build/src/bin/pg_checksums/tmp_check/t_002_actions_node_checksum_data/pgdata --filenode 16391 -v
pg_checksums: checksums verified in file "/home/bf/build/buildfarm-serinus/HEAD/pgsql.build/src/bin/pg_checksums/tmp_check/t_002_actions_node_checksum_data/pgdata/pg_tblspc/16387/PG_15_202203241/5/16391"
Checksum operation completed
Files scanned: 1
Blocks scanned: 45
Bad checksums: 0
Data checksum version: 1
If I twiddle further bits, I see that page failing checksum verification, as
expected.
I made the script copy the file before twiddling it around:
00000000 00 00 00 00 C0 01 5B 01 16 7D 00 00 A0 03 C0 03 00 20 04 20 00 00 00 00 E0 9F 38 00 C0 9F 38 00 ......[..}....... . ......8...8.
00000020 A0 9F 38 00 80 9F 38 00 60 9F 38 00 40 9F 38 00 20 9F 38 00 00 9F 38 00 E0 9E 38 00 C0 9E 38 00 ..8...8.`[email protected]. .8...8...8...8.
So it's indeed modified.
The only thing I can really conclude here is that we apparently end up with
the same checksum for exactly the modifications we are doing? Just on those
two damn instances? Reliably?
Gotta make some food. Suggestions what exactly to look at welcome.
Greetings,
Andres Freund
PS: I should really rename the hostname of that machine one of these days...
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 04:08 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 04:08 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Andres Freund <[email protected]> writes:
> The only thing I can really conclude here is that we apparently end up with
> the same checksum for exactly the modifications we are doing? Just on those
> two damn instances? Reliably?
IIRC, the table's OID or relfilenode enters into the checksum.
Could it be that assigning a specific OID to the table allows
this to happen, and these two animals are somehow assigning
that OID while others are using some slightly different OID?
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 04:54 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Andres Freund @ 2022-03-25 04:54 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-25 00:08:20 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > The only thing I can really conclude here is that we apparently end up with
> > the same checksum for exactly the modifications we are doing? Just on those
> > two damn instances? Reliably?
>
> IIRC, the table's OID or relfilenode enters into the checksum.
> Could it be that assigning a specific OID to the table allows
> this to happen, and these two animals are somehow assigning
> that OID while others are using some slightly different OID?
It's just the block number that goes into it.
I do see that the LSN that ends up on the page is the same across a few runs
of the test on serinus. Which presumably differs between different
animals. Surprised that it's this predictable - but I guess the run is short
enough that there's no variation due to autovacuum, checkpoints etc.
If I add a 'SELECT txid_current()' before the CREATE TABLE in
check_relation_corruption(), the test doesn't fail anymore, because there's an
additional WAL record.
16bit checksums for the win.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 05:23 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 05:23 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Andres Freund <[email protected]> writes:
> I do see that the LSN that ends up on the page is the same across a few runs
> of the test on serinus. Which presumably differs between different
> animals. Surprised that it's this predictable - but I guess the run is short
> enough that there's no variation due to autovacuum, checkpoints etc.
Uh-huh. I'm not surprised that it's repeatable on a given animal.
What remains to be explained:
1. Why'd it start failing now? I'm guessing that ce95c5437 *was* the
culprit after all, by slightly changing the amount of catalog data
written during initdb, and thus moving the initial LSN.
2. Why just these two animals? If initial LSN is the critical thing,
then the results of "locale -a" would affect it, so platform
dependence is hardly surprising ... but I'd have thought that all
the animals on that host would use the same initial set of
collations. OTOH, I see petalura and pogona just fell over too.
Do you have some of those animals --with-icu and others not?
> 16bit checksums for the win.
Yay :-(
As for a fix, would damaging more of the page help? I guess
it'd just move around the one-in-64K chance of failure.
Maybe we have to intentionally corrupt (e.g. invert) the
checksum field specifically.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 05:26 Andres Freund <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 05:26 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-24 21:54:38 -0700, Andres Freund wrote:
> I do see that the LSN that ends up on the page is the same across a few runs
> of the test on serinus. Which presumably differs between different
> animals. Surprised that it's this predictable - but I guess the run is short
> enough that there's no variation due to autovacuum, checkpoints etc.
This actually explains how the issue could start to be visible with
ce95c543763. It changes the amount of WAL initdb generates and therefore
influences what LSN the page ends up with. I've verified that the failing
test is reproducible with ce95c543763, but not its parent 7dac61402e3. While
of course ce95c543763 isn't "actually responsible".
Ah, and that's finally also the explanation why I couldn't reproduce the
failure it in a different directory, with an otherwise identically configured
PG: The length of the path to the tablespace influences the size of the
XLOG_TBLSPC_CREATE record.
Not sure what to do here... I guess we can just change the value we overwrite
the page with and hope to not hit this again? But that feels deeply deeply
unsatisfying.
Perhaps it would be enough to write into multiple parts of the page? I am very
much not a cryptographical expert, but the way pg_checksum_block() works, it
looks to me that "multiple" changes within a 16 byte chunk have a smaller
influence on the overall result than the same "amount" of changes to separate
16 byte chunks.
I might have to find a store still selling strong beverages at this hour.
- Andres
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 05:34 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Andres Freund @ 2022-03-25 05:34 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-25 01:23:00 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > I do see that the LSN that ends up on the page is the same across a few runs
> > of the test on serinus. Which presumably differs between different
> > animals. Surprised that it's this predictable - but I guess the run is short
> > enough that there's no variation due to autovacuum, checkpoints etc.
>
> Uh-huh. I'm not surprised that it's repeatable on a given animal.
> What remains to be explained:
>
> 1. Why'd it start failing now? I'm guessing that ce95c5437 *was* the
> culprit after all, by slightly changing the amount of catalog data
> written during initdb, and thus moving the initial LSN.
Yep, verified that (see mail I just sent).
> 2. Why just these two animals? If initial LSN is the critical thing,
> then the results of "locale -a" would affect it, so platform
> dependence is hardly surprising ... but I'd have thought that all
> the animals on that host would use the same initial set of
> collations.
I think it's the animal's name that makes the difference, due to the
tablespace path lenght thing. And while I was confused for a second by
petalura
pogona
serinus
dragonet
failing, despite different name lengths, it still makes sense: We MAXALIGN the
start of records. Which explains why flaviventris didn't fail the same way.
> As for a fix, would damaging more of the page help? I guess
> it'd just move around the one-in-64K chance of failure.
As I wrote in the other email, I think spreading the changes out wider might
help. But it's still not great. However:
> Maybe we have to intentionally corrupt (e.g. invert) the
> checksum field specifically.
seems like it'd do the trick? Even a single bit change of the checksum ought
to do, as long as we don't set it to 0.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 05:38 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 05:38 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Andres Freund <[email protected]> writes:
> Ah, and that's finally also the explanation why I couldn't reproduce the
> failure it in a different directory, with an otherwise identically configured
> PG: The length of the path to the tablespace influences the size of the
> XLOG_TBLSPC_CREATE record.
Ooooohhh ... yeah, that could explain a lot of cross-animal variation.
> Not sure what to do here... I guess we can just change the value we overwrite
> the page with and hope to not hit this again? But that feels deeply deeply
> unsatisfying.
AFAICS, this strategy of whacking a predetermined chunk of the page with
a predetermined value is going to fail 1-out-of-64K times. We have to
change the test so that it's guaranteed to produce an invalid checksum.
Inverting just the checksum field, without doing anything else, would
do that ... but that feels pretty unsatisfying too.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 06:07 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Andres Freund @ 2022-03-25 06:07 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-25 01:38:45 -0400, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > Not sure what to do here... I guess we can just change the value we overwrite
> > the page with and hope to not hit this again? But that feels deeply deeply
> > unsatisfying.
>
> AFAICS, this strategy of whacking a predetermined chunk of the page with
> a predetermined value is going to fail 1-out-of-64K times.
Yea. I suspect that the way the modifications and checksumming are done are
actually higher chance than 1/64k. But even it actually is 1/64k, it's not
great to wait for (#animals * #catalog-changes) to approach a decent
percentage of 1/64k.
I'm was curious whether there have been similar issues in the past. Querying
the buildfarm logs suggests not, at least not in the pg_checksums test.
> We have to change the test so that it's guaranteed to produce an invalid
> checksum. Inverting just the checksum field, without doing anything else,
> would do that ... but that feels pretty unsatisfying too.
We really ought to find a way to get to wider checksums :/
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 13:22 Robert Haas <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Robert Haas @ 2022-03-25 13:22 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 2:07 AM Andres Freund <[email protected]> wrote:
> We really ought to find a way to get to wider checksums :/
Eh, let's just use longer names for the buildfarm animals and call it good. :-)
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 13:49 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 13:49 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Andres Freund <[email protected]> writes:
> On 2022-03-25 01:38:45 -0400, Tom Lane wrote:
>> AFAICS, this strategy of whacking a predetermined chunk of the page with
>> a predetermined value is going to fail 1-out-of-64K times.
> Yea. I suspect that the way the modifications and checksumming are done are
> actually higher chance than 1/64k. But even it actually is 1/64k, it's not
> great to wait for (#animals * #catalog-changes) to approach a decent
> percentage of 1/64k.
Exactly.
> I'm was curious whether there have been similar issues in the past. Querying
> the buildfarm logs suggests not, at least not in the pg_checksums test.
That test has only been there since 2018 (b34e84f16). We've probably
accumulated a couple hundred initial-catalog-contents changes since
then, so maybe this failure arrived right on schedule :-(.
> We really ought to find a way to get to wider checksums :/
That'll just reduce the probability of failure, not eliminate it.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 13:53 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Robert Haas @ 2022-03-25 13:53 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 9:49 AM Tom Lane <[email protected]> wrote:
> That'll just reduce the probability of failure, not eliminate it.
I mean, if the expected time to the first failure on even 1 machine
exceeds the time until the heat death of the universe by 10 orders of
magnitude, it's probably good enough.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 14:02 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Tom Lane @ 2022-03-25 14:02 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> On Fri, Mar 25, 2022 at 9:49 AM Tom Lane <[email protected]> wrote:
>> That'll just reduce the probability of failure, not eliminate it.
> I mean, if the expected time to the first failure on even 1 machine
> exceeds the time until the heat death of the universe by 10 orders of
> magnitude, it's probably good enough.
Adding another 16 bits won't get you to that, sadly. Yeah, it *might*
extend the MTTF to more than the project's likely lifespan, but that
doesn't mean we couldn't get unlucky next week.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 14:13 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Robert Haas @ 2022-03-25 14:13 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 10:02 AM Tom Lane <[email protected]> wrote:
> Robert Haas <[email protected]> writes:
> > On Fri, Mar 25, 2022 at 9:49 AM Tom Lane <[email protected]> wrote:
> >> That'll just reduce the probability of failure, not eliminate it.
>
> > I mean, if the expected time to the first failure on even 1 machine
> > exceeds the time until the heat death of the universe by 10 orders of
> > magnitude, it's probably good enough.
>
> Adding another 16 bits won't get you to that, sadly. Yeah, it *might*
> extend the MTTF to more than the project's likely lifespan, but that
> doesn't mean we couldn't get unlucky next week.
I suspect that the number of bits Andres wants to add is no less than 48.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 14:34 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
0 siblings, 2 replies; 94+ messages in thread
From: Tom Lane @ 2022-03-25 14:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> On Fri, Mar 25, 2022 at 10:02 AM Tom Lane <[email protected]> wrote:
>> Adding another 16 bits won't get you to that, sadly. Yeah, it *might*
>> extend the MTTF to more than the project's likely lifespan, but that
>> doesn't mean we couldn't get unlucky next week.
> I suspect that the number of bits Andres wants to add is no less than 48.
I dunno. Compatibility and speed concerns aside, that seems like an awful
lot of bits to be expending on every page compared to the value.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 14:49 Robert Haas <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 2 replies; 94+ messages in thread
From: Robert Haas @ 2022-03-25 14:49 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 10:34 AM Tom Lane <[email protected]> wrote:
> I dunno. Compatibility and speed concerns aside, that seems like an awful
> lot of bits to be expending on every page compared to the value.
I dunno either, but over on the TDE thread people seemed quite willing
to expend like 16-32 *bytes* for page verifiers and nonces and things.
For compatibility and speed reasons, I doubt we could ever get by with
doing that in every cluster, but I do have some hope of introducing
something like that someday at least as an optional feature. It's not
like a 16-bit checksum was state-of-the-art even when we introduced
it. We just did it because we had 2 bytes that we could repurpose
relatively painlessly, and not any larger number. And that's still the
case today, so at least in the short term we will have to choose some
other solution to this problem.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 15:50 Tom Lane <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 2 replies; 94+ messages in thread
From: Tom Lane @ 2022-03-25 15:50 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Robert Haas <[email protected]> writes:
> ... It's not
> like a 16-bit checksum was state-of-the-art even when we introduced
> it. We just did it because we had 2 bytes that we could repurpose
> relatively painlessly, and not any larger number. And that's still the
> case today, so at least in the short term we will have to choose some
> other solution to this problem.
Indeed. I propose the attached, which also fixes the unsafe use
of seek() alongside syswrite(), directly contrary to what "man perlfunc"
says to do.
regards, tom lane
Attachments:
[text/x-diff] make-checksum-corruption-more-reliable.patch (1.4K, ../../[email protected]/2-make-checksum-corruption-more-reliable.patch)
download | inline diff:
diff --git a/src/bin/pg_checksums/t/002_actions.pl b/src/bin/pg_checksums/t/002_actions.pl
index 62c608eaf6..8c70453a45 100644
--- a/src/bin/pg_checksums/t/002_actions.pl
+++ b/src/bin/pg_checksums/t/002_actions.pl
@@ -24,6 +24,7 @@ sub check_relation_corruption
my $tablespace = shift;
my $pgdata = $node->data_dir;
+ # Create table and discover its filesystem location.
$node->safe_psql(
'postgres',
"CREATE TABLE $table AS SELECT a FROM generate_series(1,10000) AS a;
@@ -37,9 +38,6 @@ sub check_relation_corruption
my $relfilenode_corrupted = $node->safe_psql('postgres',
"SELECT relfilenode FROM pg_class WHERE relname = '$table';");
- # Set page header and block size
- my $pageheader_size = 24;
- my $block_size = $node->safe_psql('postgres', 'SHOW block_size;');
$node->stop;
# Checksums are correct for single relfilenode as the table is not
@@ -55,8 +53,12 @@ sub check_relation_corruption
# Time to create some corruption
open my $file, '+<', "$pgdata/$file_corrupted";
- seek($file, $pageheader_size, SEEK_SET);
- syswrite($file, "\0\0\0\0\0\0\0\0\0");
+ my $pageheader;
+ sysread($file, $pageheader, 24) or die "sysread failed";
+ # This inverts the pd_checksum field (only); see struct PageHeaderData
+ $pageheader ^= "\0\0\0\0\0\0\0\0\xff\xff";
+ sysseek($file, 0, 0) or die "sysseek failed";
+ syswrite($file, $pageheader) or die "syswrite failed";
close $file;
# Checksum checks on single relfilenode fail
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 16:11 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Andres Freund @ 2022-03-25 16:11 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Hi,
On 2022-03-25 11:50:48 -0400, Tom Lane wrote:
> Robert Haas <[email protected]> writes:
> > ... It's not
> > like a 16-bit checksum was state-of-the-art even when we introduced
> > it. We just did it because we had 2 bytes that we could repurpose
> > relatively painlessly, and not any larger number. And that's still the
> > case today, so at least in the short term we will have to choose some
> > other solution to this problem.
>
> Indeed. I propose the attached, which also fixes the unsafe use
> of seek() alongside syswrite(), directly contrary to what "man perlfunc"
> says to do.
That looks reasonable. Although I wonder if we loose something by not testing
the influence of the rest of the block - but I don't really see anything.
The same code also exists in src/bin/pg_basebackup/t/010_pg_basebackup.pl,
which presumably has the same collision risks. Perhaps we should put a
function into Cluster.pm and use it from both?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 16:20 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Tom Lane @ 2022-03-25 16:20 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Robert Haas <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Andres Freund <[email protected]> writes:
> The same code also exists in src/bin/pg_basebackup/t/010_pg_basebackup.pl,
> which presumably has the same collision risks.
Oooh, I missed that.
> Perhaps we should put a
> function into Cluster.pm and use it from both?
+1, I'll make it so.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 16:26 Dagfinn Ilmari Mannsåker <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Dagfinn Ilmari Mannsåker @ 2022-03-25 16:26 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Tom Lane <[email protected]> writes:
> Robert Haas <[email protected]> writes:
>> ... It's not
>> like a 16-bit checksum was state-of-the-art even when we introduced
>> it. We just did it because we had 2 bytes that we could repurpose
>> relatively painlessly, and not any larger number. And that's still the
>> case today, so at least in the short term we will have to choose some
>> other solution to this problem.
>
> Indeed. I propose the attached, which also fixes the unsafe use
> of seek() alongside syswrite(), directly contrary to what "man perlfunc"
> says to do.
LGTM, but it would be good to include $! in the die messages.
- ilmari
> regards, tom lane
>
> diff --git a/src/bin/pg_checksums/t/002_actions.pl b/src/bin/pg_checksums/t/002_actions.pl
> index 62c608eaf6..8c70453a45 100644
> --- a/src/bin/pg_checksums/t/002_actions.pl
> +++ b/src/bin/pg_checksums/t/002_actions.pl
> @@ -24,6 +24,7 @@ sub check_relation_corruption
> my $tablespace = shift;
> my $pgdata = $node->data_dir;
>
> + # Create table and discover its filesystem location.
> $node->safe_psql(
> 'postgres',
> "CREATE TABLE $table AS SELECT a FROM generate_series(1,10000) AS a;
> @@ -37,9 +38,6 @@ sub check_relation_corruption
> my $relfilenode_corrupted = $node->safe_psql('postgres',
> "SELECT relfilenode FROM pg_class WHERE relname = '$table';");
>
> - # Set page header and block size
> - my $pageheader_size = 24;
> - my $block_size = $node->safe_psql('postgres', 'SHOW block_size;');
> $node->stop;
>
> # Checksums are correct for single relfilenode as the table is not
> @@ -55,8 +53,12 @@ sub check_relation_corruption
>
> # Time to create some corruption
> open my $file, '+<', "$pgdata/$file_corrupted";
> - seek($file, $pageheader_size, SEEK_SET);
> - syswrite($file, "\0\0\0\0\0\0\0\0\0");
> + my $pageheader;
> + sysread($file, $pageheader, 24) or die "sysread failed";
> + # This inverts the pd_checksum field (only); see struct PageHeaderData
> + $pageheader ^= "\0\0\0\0\0\0\0\0\xff\xff";
> + sysseek($file, 0, 0) or die "sysseek failed";
> + syswrite($file, $pageheader) or die "syswrite failed";
> close $file;
>
> # Checksum checks on single relfilenode fail
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-25 17:31 Tom Lane <[email protected]>
parent: Dagfinn Ilmari Mannsåker <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Tom Lane @ 2022-03-25 17:31 UTC (permalink / raw)
To: Dagfinn Ilmari Mannsåker <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= <[email protected]> writes:
> LGTM, but it would be good to include $! in the die messages.
Roger, will do.
regards, tom lane
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-26 06:03 Michael Paquier <[email protected]>
parent: Tom Lane <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Michael Paquier @ 2022-03-26 06:03 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Robert Haas <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Fri, Mar 25, 2022 at 10:34:49AM -0400, Tom Lane wrote:
> Robert Haas <[email protected]> writes:
>> On Fri, Mar 25, 2022 at 10:02 AM Tom Lane <[email protected]> wrote:
>>> Adding another 16 bits won't get you to that, sadly. Yeah, it *might*
>>> extend the MTTF to more than the project's likely lifespan, but that
>>> doesn't mean we couldn't get unlucky next week.
>
>> I suspect that the number of bits Andres wants to add is no less than 48.
>
> I dunno. Compatibility and speed concerns aside, that seems like an awful
> lot of bits to be expending on every page compared to the value.
Err. And there are not that many bits that could be recycled for this
purpose in the current page layout, aren't there?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-28 00:59 Kyotaro Horiguchi <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 94+ messages in thread
From: Kyotaro Horiguchi @ 2022-03-28 00:59 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; pgsql-hackers; [email protected]
At Thu, 24 Mar 2022 15:33:29 -0400, Robert Haas <[email protected]> wrote in
> On Thu, Mar 17, 2022 at 9:21 PM Kyotaro Horiguchi
> <[email protected]> wrote:
> > All versions pass check world.
>
> Thanks, committed.
(I was overwhelmed by the flood of following discussion..)
Anyway, thanks for picking up this and committing!
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-29 16:34 Stephen Frost <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 1 reply; 94+ messages in thread
From: Stephen Frost @ 2022-03-29 16:34 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
Greetings,
* Robert Haas ([email protected]) wrote:
> On Fri, Mar 25, 2022 at 10:34 AM Tom Lane <[email protected]> wrote:
> > I dunno. Compatibility and speed concerns aside, that seems like an awful
> > lot of bits to be expending on every page compared to the value.
>
> I dunno either, but over on the TDE thread people seemed quite willing
> to expend like 16-32 *bytes* for page verifiers and nonces and things.
Absolutely.
> For compatibility and speed reasons, I doubt we could ever get by with
> doing that in every cluster, but I do have some hope of introducing
> something like that someday at least as an optional feature. It's not
> like a 16-bit checksum was state-of-the-art even when we introduced
> it. We just did it because we had 2 bytes that we could repurpose
> relatively painlessly, and not any larger number. And that's still the
> case today, so at least in the short term we will have to choose some
> other solution to this problem.
I agree that this would be great as an optional feature. Those patches
to allow the system to be built with reserved space for $whatever would
allow us to have a larger checksum for those who want it and perhaps a
nonce with TDE for those who wish that in the future. I mentioned
before that I thought it might be a good way to introduce page-level
epochs for 64bit xids too though it never seemed to get much traction.
Anyhow, this whole thread has struck me as a good reason to polish those
patches off and add on top of them an extended checksum ability, first,
independent of TDE, and remove the dependency of those patches from the
TDE effort and instead allow it to just leverage that ability. I still
suspect we'll have some folks who will want TDE w/o a per-page nonce and
that could be an option but we'd be able to support TDE w/ integrity
pretty easily, which would be fantastic.
Thanks,
Stephen
Attachments:
[application/pgp-signature] signature.asc (819B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: Corruption during WAL replay
@ 2022-03-29 17:04 Robert Haas <[email protected]>
parent: Stephen Frost <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Robert Haas @ 2022-03-29 17:04 UTC (permalink / raw)
To: Stephen Frost <[email protected]>; +Cc: Tom Lane <[email protected]>; Andres Freund <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; Ibrar Ahmed <[email protected]>; [email protected]; hlinnaka <[email protected]>; Masahiko Sawada <[email protected]>; pgsql-hackers; Daniel Wood <[email protected]>
On Tue, Mar 29, 2022 at 12:34 PM Stephen Frost <[email protected]> wrote:
> Anyhow, this whole thread has struck me as a good reason to polish those
> patches off and add on top of them an extended checksum ability, first,
> independent of TDE, and remove the dependency of those patches from the
> TDE effort and instead allow it to just leverage that ability. I still
> suspect we'll have some folks who will want TDE w/o a per-page nonce and
> that could be an option but we'd be able to support TDE w/ integrity
> pretty easily, which would be fantastic.
Yes, I like that idea. Once we get beyond feature freeze, perhaps we
can try to coordinate to avoid duplication of effort -- or absence of
effort.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: pg_rewind fails with in-place tablespace
@ 2023-07-25 07:36 Michael Paquier <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Michael Paquier @ 2023-07-25 07:36 UTC (permalink / raw)
To: 赵锐(惜元) <[email protected]>; +Cc: pgsql-hackers <[email protected]>; Thomas Munro <[email protected]>
On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
> Recently I encountered an issue: pg_rewind fails when dealing with
> in-place tablespace. The problem seems to be that pg_rewind is
> treating in-place tablespace as symbolic link, while in fact it
> should be treated as directory.
> Here is the output of pg_rewind:
> pg_rewind: error: file "pg_tblspc/16385" is of different type in
> source and target
> To help reproduce the failure, I have attached a tap test. And I am
> pleased to say that I have also identified a solution for this
> problem, which I have included in the patch.
> Thank you for your attention to this matter.
Issue reproduced here, and agreed that we'd better do something about
that. I am not sure if your patch is right for the job though, but
I'll try to study that a bit more.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: pg_rewind fails with in-place tablespace
@ 2023-07-28 07:54 Michael Paquier <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Michael Paquier @ 2023-07-28 07:54 UTC (permalink / raw)
To: 赵锐(惜元) <[email protected]>; +Cc: pgsql-hackers <[email protected]>; Thomas Munro <[email protected]>
On Tue, Jul 25, 2023 at 04:36:42PM +0900, Michael Paquier wrote:
> On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
>> To help reproduce the failure, I have attached a tap test. And I am
>> pleased to say that I have also identified a solution for this
>> problem, which I have included in the patch.
>> Thank you for your attention to this matter.
>
> Issue reproduced here, and agreed that we'd better do something about
> that. I am not sure if your patch is right for the job though, but
> I'll try to study that a bit more.
It took me some time to remember that for the case of a local source
we'd finish by using recurse_dir() and consider the in-place
tablespace as a regular directory, so a fix located in
libpq_traverse_files() sounds good to me.
+ if (strncmp(link_target, "pg_tblspc/", strlen("pg_tblspc/")) == 0)
+ type = FILE_TYPE_DIRECTORY;
+ else
+ type = FILE_TYPE_SYMLINK;
However this is not consistent with the other places where we detect
if an in-place tablespace is used, like pg_basebackup.c, where we rely
on the fact that the tablespace path is a relative path, using
is_absolute_path() to make the difference between a normal and
in-place tablespace. I would choose consistency and do the same here,
checking if we have an absolute or relative path, depending on the
result of pg_tablespace_location().
Testing only for the creation of the tablespace is fine for the sake
of the report, but I would slightly more here and create a table on
this tablespace with some data, and a check_query() once pg_rewind is
done.
I am finishing with the attached. Thoughts?
--
Michael
Attachments:
[text/x-diff] 0001-Fix-pg_rewind-with-in-place-tablespaces-when-source-.patch (3.0K, ../../[email protected]/2-0001-Fix-pg_rewind-with-in-place-tablespaces-when-source-.patch)
download | inline diff:
From 728bd093959f0d51c234f88729798165176fc0f6 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Fri, 28 Jul 2023 16:53:21 +0900
Subject: [PATCH] Fix pg_rewind with in-place tablespaces when source is remote
---
src/bin/pg_rewind/libpq_source.c | 11 ++++++++++-
src/bin/pg_rewind/t/001_basic.pl | 20 ++++++++++++++++++++
src/bin/pg_rewind/t/RewindTest.pm | 1 +
3 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/src/bin/pg_rewind/libpq_source.c b/src/bin/pg_rewind/libpq_source.c
index 0d8e9ee2d1..417c74cfef 100644
--- a/src/bin/pg_rewind/libpq_source.c
+++ b/src/bin/pg_rewind/libpq_source.c
@@ -298,7 +298,16 @@ libpq_traverse_files(rewind_source *source, process_file_callback_t callback)
link_target = PQgetvalue(res, i, 3);
if (link_target[0])
- type = FILE_TYPE_SYMLINK;
+ {
+ /*
+ * In-place tablespaces are directories located in pg_tblspc/ with
+ * relative paths.
+ */
+ if (is_absolute_path(link_target))
+ type = FILE_TYPE_SYMLINK;
+ else
+ type = FILE_TYPE_DIRECTORY;
+ }
else if (isdir)
type = FILE_TYPE_DIRECTORY;
else
diff --git a/src/bin/pg_rewind/t/001_basic.pl b/src/bin/pg_rewind/t/001_basic.pl
index 031594e14e..c7b48255a7 100644
--- a/src/bin/pg_rewind/t/001_basic.pl
+++ b/src/bin/pg_rewind/t/001_basic.pl
@@ -18,6 +18,12 @@ sub run_test
RewindTest::setup_cluster($test_mode);
RewindTest::start_primary();
+ # Create an in-place tablespace with some data on it.
+ primary_psql("CREATE TABLESPACE space_test LOCATION ''");
+ primary_psql("CREATE TABLE space_tbl (d text) TABLESPACE space_test");
+ primary_psql(
+ "INSERT INTO space_tbl VALUES ('in primary, before promotion')");
+
# Create a test table and insert a row in primary.
primary_psql("CREATE TABLE tbl1 (d text)");
primary_psql("INSERT INTO tbl1 VALUES ('in primary')");
@@ -78,6 +84,13 @@ sub run_test
"insert into drop_tbl values ('in primary, after promotion')");
primary_psql("DROP TABLE drop_tbl");
+ # Insert some data in the in-place tablespace for the old primary and
+ # the standby.
+ primary_psql(
+ "INSERT INTO space_tbl VALUES ('in primary, after promotion')");
+ standby_psql(
+ "INSERT INTO space_tbl VALUES ('in standby, after promotion')");
+
# Before running pg_rewind, do a couple of extra tests with several
# option combinations. As the code paths taken by those tests
# do not change for the "local" and "remote" modes, just run them
@@ -145,6 +158,13 @@ sub run_test
RewindTest::run_pg_rewind($test_mode);
+ check_query(
+ 'SELECT * FROM space_tbl ORDER BY d',
+ qq(in primary, before promotion
+in standby, after promotion
+),
+ 'table content');
+
check_query(
'SELECT * FROM tbl1',
qq(in primary
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 4957791e94..8fbbd521cb 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -131,6 +131,7 @@ sub setup_cluster
$node_primary->append_conf(
'postgresql.conf', qq(
wal_keep_size = 320MB
+allow_in_place_tablespaces = on
));
return;
}
--
2.40.1
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* Re: pg_rewind fails with in-place tablespace
@ 2023-07-30 22:48 Michael Paquier <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 94+ messages in thread
From: Michael Paquier @ 2023-07-30 22:48 UTC (permalink / raw)
To: 赵锐(惜元) <[email protected]>; +Cc: pgsql-hackers <[email protected]>; Thomas Munro <[email protected]>
On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached. Thoughts?
Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers. If there are opinions in favor
of a backpatch, feel free of course.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 94+ messages in thread
* 回复：pg_rewind fails with in-place tablespace
@ 2023-07-31 02:07 Rui Zhao <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 0 replies; 94+ messages in thread
From: Rui Zhao @ 2023-07-31 02:07 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: pgsql-hackers <[email protected]>; Thomas Munro <[email protected]>
Sorry for the delay in responding to this matter as I have been waiting for another similar subject to approved by a moderator.
Upon review, I am satisfied with the proposed solution and believe that checking absolute path is better than hard coding with "pg_tblspc/". I think we have successfully resolved this issue in the pg_rewind case.
However, I would like to bring your attention to another issue: pg_upgrade fails with in-place tablespace. Another issue is still waiting for approved. I have tested all the tools in src/bin with in-place tablespace, and I believe this is the final issue.
Thank you for your understanding and assistance.
Best regard,
Rui Zhao
------------------------------------------------------------------
发件人：Michael Paquier <[email protected]>
发送时间：2023年7月31日(星期一) 06:49
收件人：赵锐(惜元) <[email protected]>
抄 送：pgsql-hackers <[email protected]>; Thomas Munro <[email protected]>
主 题：Re: pg_rewind fails with in-place tablespace
On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached. Thoughts?
Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers. If there are opinions in favor
of a backpatch, feel free of course.
--
Michael
^ permalink raw reply [nested|flat] 94+ messages in thread
end of thread, other threads:[~2023-07-31 02:07 UTC | newest]
Thread overview: 94+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-23 20:56 Corruption during WAL replay Teja Mupparti <[email protected]>
2020-03-24 09:18 ` Kyotaro Horiguchi <[email protected]>
2020-03-30 23:31 ` Andres Freund <[email protected]>
2020-03-31 07:36 ` Kyotaro Horiguchi <[email protected]>
2020-04-10 23:59 ` Teja Mupparti <[email protected]>
2020-04-13 06:24 ` Masahiko Sawada <[email protected]>
2020-04-13 08:40 ` Andres Freund <[email protected]>
2020-04-13 09:53 ` Masahiko Sawada <[email protected]>
2020-04-14 02:35 ` Kyotaro Horiguchi <[email protected]>
2020-04-14 19:04 ` Teja Mupparti <[email protected]>
2020-06-12 08:20 ` Masahiko Sawada <[email protected]>
2020-08-17 11:05 ` Heikki Linnakangas <[email protected]>
2020-08-17 18:22 ` Andres Freund <[email protected]>
2020-10-30 16:34 ` Anastasia Lubennikova <[email protected]>
2020-11-06 11:40 ` Masahiko Sawada <[email protected]>
2020-12-01 14:58 ` Anastasia Lubennikova <[email protected]>
2021-01-06 08:33 ` Kyotaro Horiguchi <[email protected]>
2021-03-04 17:37 ` Ibrar Ahmed <[email protected]>
2021-03-05 03:01 ` Kyotaro Horiguchi <[email protected]>
2021-08-10 18:14 ` Robert Haas <[email protected]>
2021-09-24 19:37 ` Tom Lane <[email protected]>
2021-09-24 20:08 ` Robert Haas <[email protected]>
2021-09-24 20:22 ` Tom Lane <[email protected]>
2021-09-27 08:28 ` Kyotaro Horiguchi <[email protected]>
2021-09-27 08:30 ` Kyotaro Horiguchi <[email protected]>
2020-04-11 00:49 ` Alvaro Herrera <[email protected]>
2020-04-11 00:54 ` Andres Freund <[email protected]>
2020-12-16 10:13 [PATCH 4/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v2 1/2] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2021-01-28 16:42 [PATCH v3 2/5] Change conversion function signature. Heikki Linnakangas <[email protected]>
2022-03-24 19:33 Re: Corruption during WAL replay Robert Haas <[email protected]>
2022-03-24 22:04 ` Tom Lane <[email protected]>
2022-03-25 00:37 ` Robert Haas <[email protected]>
2022-03-25 00:39 ` Robert Haas <[email protected]>
2022-03-25 00:45 ` Tom Lane <[email protected]>
2022-03-25 01:08 ` Robert Haas <[email protected]>
2022-03-25 01:22 ` Tom Lane <[email protected]>
2022-03-25 01:59 ` Tom Lane <[email protected]>
2022-03-25 02:20 ` Andres Freund <[email protected]>
2022-03-25 02:43 ` Andres Freund <[email protected]>
2022-03-25 03:43 ` Andres Freund <[email protected]>
2022-03-25 04:08 ` Tom Lane <[email protected]>
2022-03-25 04:54 ` Andres Freund <[email protected]>
2022-03-25 05:23 ` Tom Lane <[email protected]>
2022-03-25 05:34 ` Andres Freund <[email protected]>
2022-03-25 05:26 ` Andres Freund <[email protected]>
2022-03-25 05:38 ` Tom Lane <[email protected]>
2022-03-25 06:07 ` Andres Freund <[email protected]>
2022-03-25 13:22 ` Robert Haas <[email protected]>
2022-03-25 13:49 ` Tom Lane <[email protected]>
2022-03-25 13:53 ` Robert Haas <[email protected]>
2022-03-25 14:02 ` Tom Lane <[email protected]>
2022-03-25 14:13 ` Robert Haas <[email protected]>
2022-03-25 14:34 ` Tom Lane <[email protected]>
2022-03-25 14:49 ` Robert Haas <[email protected]>
2022-03-25 15:50 ` Tom Lane <[email protected]>
2022-03-25 16:11 ` Andres Freund <[email protected]>
2022-03-25 16:20 ` Tom Lane <[email protected]>
2022-03-25 16:26 ` Dagfinn Ilmari Mannsåker <[email protected]>
2022-03-25 17:31 ` Tom Lane <[email protected]>
2022-03-29 16:34 ` Stephen Frost <[email protected]>
2022-03-29 17:04 ` Robert Haas <[email protected]>
2022-03-26 06:03 ` Michael Paquier <[email protected]>
2022-03-25 02:14 ` Andres Freund <[email protected]>
2022-03-25 02:23 ` Thomas Munro <[email protected]>
2022-03-25 02:35 ` Andres Freund <[email protected]>
2022-03-25 03:02 ` Thomas Munro <[email protected]>
2022-03-25 01:23 ` Andres Freund <[email protected]>
2022-03-28 00:59 ` Kyotaro Horiguchi <[email protected]>
2023-07-25 07:36 Re: pg_rewind fails with in-place tablespace Michael Paquier <[email protected]>
2023-07-28 07:54 ` Re: pg_rewind fails with in-place tablespace Michael Paquier <[email protected]>
2023-07-30 22:48 ` Re: pg_rewind fails with in-place tablespace Michael Paquier <[email protected]>
2023-07-31 02:07 ` 回复：pg_rewind fails with in-place tablespace Rui Zhao <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox