Subject: pgbench with libevent?

From: Fabien COELHO <[email-masked]>
Date: Sun, 13 Aug 2023 11:32:08 +0200 (CEST)

Hello devs,

Pgbench is managing clients I/Os manually with select or poll. Much of 
this could be managed by libevent.

Pros:

1. libevent is portable, stable, and widely used (eg Chromium, Memcached, PgBouncer).

2. libevent implements more I/O wait methods, which may be more efficient on some platforms
    (eg FreeBSD kqueue, Windows wepoll in libevent 2.2 alpha), and hides portability issues.

3. it would remove significant portions of unattractive pgbench code, esp. in threadRun,
    and around socket/poll abstraction and portability layer.

4. depending on the number of event loops, the client load could be shared more evenly.
    currently a thread only manages its own clients, some client I/Os may be waiting to be
    processed while other threads could be available to process them.

Cons:

1. it adds a libevent dependency to postgres. This may be a no go from the start.

2. this is a significant refactoring project which implies a new internal architecture and adds
    new code to process and generate appropriate events.

3. libevent ability to function efficiently in a highly multithreaded environment
    is unclear. Should there be one event queue which generate a shared work queue?
    or several event queues, one per thread (which would remove the sharing pro)?
    or something in between? Some experiments and configuratibility may be desirable.
    This may also have an impact on pgbench user interface and output depending on the result,
    eg there may be specialized event and worker threads, some statistics may be slightly
    different, new options may be needed…

4. libevent development seems slugish, last bugfix was published 3 years ago, version
    2.2 has been baking for years, but the development seems lively (+100 contributors).

Neutral?

1. BSD 3 clauses license.

Is pros > cons, or not? Other thoughts, pros, cons?

-- 
Fabien.

---

From: Alvaro Herrera <[email-masked]>
Date: Mon, 14 Aug 2023 11:38:07 +0200

>    2.2 has been baking for years, but the development seems lively (+100 contributors).

Ugh, I would stay away from something like that.  Would we become
hostage to an undelivering group?  No thanks.

On 2023-Aug-14, Fabien COELHO wrote:

> Or maybe libuv (used by nodejs?).

> Note: libev had no updates in 8 years.

libev or libuv?  No updates in 8 years => dead.  No way.

Reworking based on wait events as proposed downthread sounds more
promising.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual)