From: fppfurtado (@fppfurtado) <noreply+fppfurtado@github.com>
To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>
Subject: [psycopg/psycopg] issue #1504: JSON/JSONB loaders decode as UTF-8 regardless of client_encoding
Date: Thu, 19 Nov 2026 14:30:18 +0000
Message-ID: <gh-psycopg-psycopg-1504@github.com> (raw)
The `json`/`jsonb` text loaders decode the value as UTF-8 regardless of the connection's `client_encoding`. On a database whose encoding is not UTF-8 (e.g. `LATIN1`), any non-ASCII character in a JSON value raises `UnicodeDecodeError`, while the same value cast to `text` loads fine.
### Reproduce
```python
import psycopg
dsn = "host=127.0.0.1 user=postgres password=x"
with psycopg.connect(dsn, dbname="postgres", autocommit=True) as c:
c.execute("CREATE DATABASE l1 ENCODING 'LATIN1' LC_COLLATE 'C' LC_CTYPE 'C' TEMPLATE template0")
with psycopg.connect(dsn, dbname="l1") as c:
print(c.info.encoding) # iso8859-1
print(c.execute("""SELECT '{"a": "é"}'::text""").fetchone()[0]) # {"a": "é"}
c.execute("""SELECT '{"a": "é"}'::json""").fetchone()[0] # UnicodeDecodeError
```
```
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 7: invalid continuation byte
```
Same result for `jsonb`, and with both `PSYCOPG_IMPL=binary` and `PSYCOPG_IMPL=python`.
Environment: psycopg 3.3.9 (`psycopg[binary]`), libpq 18.0, PostgreSQL 16 (official `postgres:16` image), Python 3.14, Linux.
### Cause
`_JsonLoader.load()` (`psycopg/types/json.py`) passes the raw bytes to `json.loads()`, which only auto-detects UTF-8/16/32. The server sends the text in the client encoding, so for a non-UTF-8 `client_encoding` the bytes are not UTF-8. `TextLoader` decodes with the connection's encoding, which is why `::text` works.
### Possible fix
Decode with the connection encoding before parsing, when it is not UTF-8 (e.g. `data.decode(conn_encoding(self.connection))`, as `TextLoader` does), keeping the bytes fast path for UTF-8. `JsonbBinaryLoader` has the same shape (it passes `data[1:]` to `loads()`), and as far as I know the server converts binary `jsonb` to the client encoding too, so it likely needs the same treatment; I have not tested the binary path.
### Workaround
Register `TextLoader` for `json`/`jsonb` and parse the resulting `str`:
```python
from psycopg.types.string import TextLoader
for t in ("json", "jsonb"):
conn.adapters.register_loader(t, TextLoader)
```