pg.ddx.io  psycopg/psycopg GitHub issues and pull requests  
help / color / mirror / Atom feed
[psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
6+ messages / 3 participants
[nested] [flat]

* [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-17 23:33  "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
  0 siblings, 0 replies; 6+ messages in thread

From: dvarrazzo (@dvarrazzo) @ 2026-11-17 23:33 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

The `psycopg.pq` module wraps most of the libpq, but a few useful functions are missing. For Psycopg 3.4 we want to add the following.

## Pipeline

- [ ] `PQsendPipelineSync()` (libpq 17) -> `PGconn.send_pipeline_sync()`

  Like `PQpipelineSync()`, but it doesn't flush the output buffer. Several syncs can then be sent with a single write, for instance in `executemany()` in pipeline mode.

## Error messages

- [ ] `PQsetErrorVerbosity()` -> `PGconn.set_error_verbosity()`
- [ ] `PQsetErrorContextVisibility()` -> `PGconn.set_error_context_visibility()`
- [ ] `PQresultVerboseErrorMessage()` -> `PGresult.verbose_error_message()`

  They control how much detail error messages include: the SQLSTATE, the CONTEXT lines, the server source location. This needs two new enums, `pq.ErrorVerbosity` (`TERSE`, `DEFAULT`, `VERBOSE`, `SQLSTATE`) and `pq.ContextVisibility` (`NEVER`, `ERRORS`, `ALWAYS`).

## SSL information

- [ ] `PQsslAttribute()` -> `PGconn.ssl_attribute()`
- [ ] `PQsslAttributeNames()` -> `PGconn.ssl_attribute_names`

  They report details of the SSL connection, such as the protocol, the cipher and the key size. They are marked as TODO in `_pq_ctypes.py`. `PQsslStruct()` and `PQgetssl()` return OpenSSL pointers, which are not useful from Python, so they are left out.

## Results

- [ ] `PQresultMemorySize()` (libpq 12) -> `PGresult.memory_size`

  The memory used by a result. Useful to diagnose memory usage.

- [ ] `PQsetvalue()` -> `PGresult.set_value()`
- [ ] `PQcopyResult()` -> `PGresult.copy()`

  We already wrap `PQmakeEmptyPGresult()` and `PQsetResultAttrs()`, so you can build the columns of a result, but you can't fill its rows.

## Notes

- Every new method needs both the Python and the C implementation, plus a `NotSupportedError` when the libpq loaded is too old, as for the other functions that need a recent libpq.
- Once these exist, we can decide whether to use them at a higher level: `send_pipeline_sync()` in `Pipeline`, the SSL attributes in `ConnectionInfo`, the error verbosity as a connection setting.
- OAuth functionalities are being added in #1190.

## Not planned

- Functions replaced by ones we already wrap: `PQrequestCancel`, `PQencryptPassword`, `PQoidStatus`, `PQsetNoticeProcessor`, `PQinitSSL`, and the old COPY API (`PQgetline`, `PQgetlineAsync`, `PQputline`, `PQputnbytes`, `PQendcopy`).
- Functions whose results we already get another way: `PQclientEncoding` and `PQsetClientEncoding` (we read and set the `client_encoding` parameter), and `PQpingParams` (`PGconn.ping()` takes a connection string).
- Callback hooks: the SSL key password hook and the events API. They can be considered separately if there is demand.
- `PQsocketPoll()`: we have our own wait functions.


^ permalink  raw  [nested|flat] 6+ messages in thread

* Re: [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-17 23:33  "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
  4 siblings, 0 replies; 6+ messages in thread

From: dvarrazzo (@dvarrazzo) @ 2026-11-17 23:33 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

@dvarrazzo set milestone to "3.4".

^ permalink  raw  [nested|flat] 6+ messages in thread

* Re: [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-18 10:55  "Abhishek-Rath (@Abhishek-Rath)" <noreply+Abhishek-Rath@github.com>
  4 siblings, 0 replies; 6+ messages in thread

From: Abhishek-Rath (@Abhishek-Rath) @ 2026-11-18 10:55 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

Hi @dvarrazzo, I would like to work on the part of this. Could I start with `PQresultMemorySize` (Python & C implementations, tests)? If that goes well I would be happy to take the error verbosity group next.

^ permalink  raw  [nested|flat] 6+ messages in thread

* Re: [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-18 16:20  "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
  4 siblings, 0 replies; 6+ messages in thread

From: dvarrazzo (@dvarrazzo) @ 2026-11-18 16:20 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

@Abhishek-Rath sure, you are welcome to do so. Looking forward for your PR 🙂 

^ permalink  raw  [nested|flat] 6+ messages in thread

* Re: [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-18 18:39  "charlie-morrison (@charlie-morrison)" <noreply+charlie-morrison@github.com>
  4 siblings, 0 replies; 6+ messages in thread

From: charlie-morrison (@charlie-morrison) @ 2026-11-18 18:39 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

@dvarrazzo could I take `PQsetvalue()` → `PGresult.set_value()` and `PQcopyResult()` → `PGresult.copy()`? Both would get the Python and the C implementation, plus tests. That doesn't overlap with what Abhishek-Rath picked (`memory_size`, then the error verbosity group).

For `copy()` I'd expose the `PG_COPYRES_*` flags as a `pq.CopyResultFlag` IntFlag, next to `Trace`, and default to attributes + tuples. If you'd rather have a different shape, I'll follow yours.


^ permalink  raw  [nested|flat] 6+ messages in thread

* Re: [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module
@ 2026-11-19 22:51  "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
  4 siblings, 0 replies; 6+ messages in thread

From: dvarrazzo (@dvarrazzo) @ 2026-11-19 22:51 UTC (permalink / raw)
  To: psycopg/psycopg <noreply+psycopg-psycopg@github.com>

@charlie-morrison ok for me. Yes, it seems you will need to add an enum. But no, I don't think you should make the default attributes + tuples. There should be no default. Note that the function should take `int` as input, the enum would only be a convenience to assemble a value via OR.

^ permalink  raw  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2026-11-19 22:51 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 23:33 [psycopg/psycopg] issue #1500: Wrap missing libpq functions in the pq module "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
2026-11-17 23:33 ` "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
2026-11-18 10:55 ` "Abhishek-Rath (@Abhishek-Rath)" <noreply+Abhishek-Rath@github.com>
2026-11-18 16:20 ` "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>
2026-11-18 18:39 ` "charlie-morrison (@charlie-morrison)" <noreply+charlie-morrison@github.com>
2026-11-19 22:51 ` "dvarrazzo (@dvarrazzo)" <noreply+dvarrazzo@github.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox